key: cord-0745144-q742b270
authors: Alamoodi, A.H.; Zaidan, B.B.; Al-Masawa, Maimonah; Taresh, Sahar M.; Noman, Sarah; Ahmaro, Ibraheem Y.Y.; Garfan, Salem; Chen, Juliana; Ahmed, M.A.; Zaidan, A.A.; Albahri, O.S.; Aickelin, Uwe; Thamir, Noor N.; Fadhil, Julanar Ahmed; Ahmed, Esma
title: Multi-perspectives systematic review on the applications of sentiment analysis for vaccine hesitancy
date: 2021-10-16
journal: Comput Biol Med
DOI: 10.1016/j.compbiomed.2021.104957
sha: 80aa2ebf12fb4503351b8e26fdcf23c565b0aab8
doc_id: 745144
cord_uid: q742b270

A substantial impediment to widespread Coronavirus disease (COVID-19) vaccination is vaccine hesitancy. Many researchers across scientific disciplines have presented countless studies in favor of COVID-19 vaccination, but misinformation on social media could hinder vaccination efforts and increase vaccine hesitancy. Nevertheless, studying people's perceptions on social media to understand their sentiment presents a powerful medium for researchers to identify the causes of vaccine hesitancy and therefore develop appropriate public health messages and interventions. To the best of the authors' knowledge, previous studies have presented vaccine hesitancy in specific cases or within one scientific discipline (i.e., social, medical, and technological). No previous study has presented findings via sentiment analysis for multiple scientific disciplines as follows: (1) social, (2) medical, public health, and (3) technology sciences. Therefore, this research aimed to review and analyze articles related different vaccine hesitancy cases in the last 11 years and understand the application of sentiment analysis on the most important literature findings. Articles were systematically searched in Web of Science, Scopus, PubMed, IEEEXplore, ScienceDirect, and Ovid from January 1, 2010, to July 2021. A total of 30 articles were selected on the basis of inclusion and exclusion criteria. These articles were formed into a taxonomy of literature, along with challenges, motivations, and recommendations for social, medical, and public health and technology sciences. Significant patterns were identified, and opportunities were promoted towards the understanding of this phenomenon.

At the time of writing this review, in the midst of the global pandemic that has claimed millions of lives, countries are racing against the time to deliver Coronavirus disease vaccines to everyone, in hopes of curbing the spread of the disease and bringing this pandemic to an end. An urgency was placed on COVID-19 vaccine development given the severity of the disease [1] . Global industrial pharmaceuticals, scientists, and governments have dedicated their efforts to decode the virus and develop vaccines, built on years of research on vaccine production [2] . Despite the enormous efforts and vaccines' remarkable success in clinical trials and in the field in terms of safety and efficacy [3] , a significant impediment to achieving global vaccination coverage is vaccine hesitancy towards approved vaccines [4] . Concerning the magnitude of the issue of "vaccine hesitancy" and to facilitate a clear understanding of the problem's origin, development, and major threats it poses, the introduction of the present review was structured in a question-and-answer setup, beginning with a fundamental question as follows:

Vaccine hesitancy is defined as the delay in acceptance or refusal of vaccines despite the availability of vaccine services [5] . Historical records show vaccine hesitancy emerged when vaccines were first introduced, dating all the way back to the 18th century when small pox vaccine was pioneered in the aftermath of the smallpox epidemic [6, 7] . At present, an increasing number of anti-vaccination groups generally consider vaccination as risky and unessential [8] . Individuals who completely oppose vaccination are known as "anti-vax" or "anti-vaxxers." However, not all vaccine-hesitant people are "anti-vaxxers" [9] . In fact, vaccine hesitancy exists on a spectrum ranging from complete acceptance to complete rejection, with hesitant people falling somewhere in the middle of this range. In this respect, vaccine-hesitant individuals may accept some vaccines, but reject, postpone, or doubt taking others [6, 8, 10] . This ambiguity clearly dictates the need to have a comprehensive understanding of the factors that contribute to the development of vaccine hesitancy, thus leading to the following question:

From the academic literature, the different drivers of vaccine hesitancy could be grouped into three main areas:

(1) vaccine-related; (2) health system-related, and (3) individual's social attributes. For those factors related to vaccine, they may include mistrust in vaccine safety [11] and effectiveness [12, 13] , which have been identified as being the most influential factors affecting vaccine uptake [11] . Viral propagation of misinformation regarding vaccines has contributed to shaping the landscape of doubt regarding vaccination [14] . A well-known incident was when a study by Andrew Wakefield in 1998 associated measles, mumps, and rubella vaccine with autism, resulting in a huge tide of public uncertainty regarding vaccine safety [15] . The emergence of social media platforms has exacerbated public uncertainty, particularly in magnifying incidents of rare adverse events. [16] . As for the health system factors, mistrust in the health care workers [17] , governments [18] , and/or health agencies [19] all contribute to the vaccine hesitancy phenomenon. In addition, a lack of confidence in the system has critically triggered a dramatic drop in vaccine uptake in certain nations [14] . In addition, another contributing factor to growing vaccine hesitancy is the health system's lack of a holistic comprehension of the problem and the insufficient foresight of the public's attitude towards vaccine enforcement measures, which collectively may prompt adverse reaction to the implemented policies [7] . The third major factor relates to vaccine-hesitant individual attributes. In that regard, different reasons exist, including individuals' religious beliefs [12, 13, 20] , deficiency of knowledge and awareness [12, 13] , and low socioeconomic status [14] , which all have been shown to influence vaccine hesitancy. In addition, vaccine hesitancy has been demonstrated to increase in financially challenged communities [21] . Low education levels were also correlated with refusal to vaccine uptake [22] . Nevertheless, these factors undergo continuous changes that necessitate ongoing monitoring to capture those fluctuations and gain an enhanced understanding of the factors driving vaccine hesitancy. With the elaboration of the modifying factors, why vaccination hesitancy is a consequence that requires immediate attention must be explored, leading to the following question;

From practical and scientific perspectives, vaccine hesitancy has been linked to the reduced vaccine acceptance rates and the recurrence of epidemics of diseases [6] . For instance, unwillingness to take measles vaccine in regions in Europe and the US contributed to the upsurge of measles outbreaks, whereby unvaccinated individuals were making up the majority of the cases [23, 24] . Refusal of vaccine uptake against diseases, such as human papillomavirus [25] or polio virus [26, 27] , has been also reported in other countries. Vaccine hesitancy, in this sense, jeopardizes not only the hesitant individual's safety but also the safety of the entire community [28] . Population immunity, which is called "herd immunity," could only be achieved when a large proportion of the population acquires vaccination [29] . Thus, without public vaccine acceptance, all immunization efforts are evidently futile [30] . As a result, the World Health Organization (WHO) has declared vaccine hesitancy as one of the world's top 10 "global health threats" [31] . According to WHO recommendations, vaccine hesitancy assessment should be an integral part of vaccination programs [32] . They also advised that incorporating vaccine hesitancy assessment is essential to help evaluate public opinions regarding vaccines and assess behaviors [33, 34] . This advice is crucial to help unravel the barriers and determinants, shape immunization policies, and allow constant evaluation of the effectiveness of the implemented strategies to assist in the efforts of combating vaccine hesitancy [35] . Therefore, the development of valid tools to detect and measure vaccine hesitancy is paramount [34] . These tools should be able to deal with the complex, heterogeneous, and changing nature of vaccine hesitancy across time and place [36] . The following question raised provides a summary of what research is available in detecting vaccine hesitancy:

Previous studies have attempted to address the phenomenon of vaccine hesitancy thoroughly from various scientific disciplines, including cognitive [37] , psychological [38] , social [39] , demographic [40] , and cultural aspects [41] . In these types of studies, numerous efforts have been made in recent years to create tools for detecting vaccine hesitancy, yet most of these tools are mainly global questionnaires and surveys that are thoroughly validated to measure vaccine hesitancy, and they are widely accepted. However, in spite of their remarkable achievements and applications, they are not sufficient for making an effective tool to measure or detect vaccine hesitancy. Meanwhile, other areas and tools emerged from other scientific disciplines and took initiatives, not only in detecting vaccine hesitancy on its own but also assisting all the other disciplines in raising their capabilities and analytical powers to achieve their desired study target. Computer science research has also contributed remarkably to addressing previous vaccine hesitancy issues. Therefore, seeing how this scientific discipline emerged and how it has been integrated is warranted. For the purpose of identifying how previous computerscience studies contributed in relation to vaccine hesitancy and their approaches used to achieve that, the following question is raised:

In the academic literature, computer science technologies are ever present in society, offering new and emerging methods to improve vaccination coverage and being used to support various public-health responses to many wellknown concerns, such as COVID-19 and vaccine hesitancy [42] . Some of the most notable works introduced include population surveillance [43] , case identification [44] , contact tracing [45] , and evaluation of interventions on the basis of mobility data and communication with the public [44] . The integration of such technologies not only works by its own but also find its way to integrate in many health domains [46] due to their rapid responses in leveraging many resources, such as large online datasets [47] , connected devices [48] , relatively low-cost computing resources [49] , and advances in predictive analysis measures, like artificial intelligence (AI) [50] and machine learning [51] . When taking a deep look, no one could argue that computer science technologies have proven their resilience and contributions in dealing with vaccine hesitancy and addressing its issues [52] . Still, most of these technologies deal with the issue from a data analytical perspective, where an existing dataset is applied for their experiments and analytical processes. However, vaccine hesitancy, in its basic form, as an opinion might not be readily found in the form of available data for experimentation; it could also be persuaded or changed by social media [53] or even word of mouth [54] . For example, word of mouth only travels between one person to another, thus making distribution to a rumor or an opinion about vaccine on individual level low, especially when governments place restrictions on the spreading of such vaccine rumors. Meanwhile, this limits an individual's freedom of speech. Social media platforms could enable the dissemination of real opinions, but the significance of rumors spreading is more difficult to monitor and handle. Computer technologies could contribute to monitoring social media and understanding the opinions of vaccination by utilizing natural language processing (NLP). In that context, understanding this branch of computer technology is important. In particular, sentiment analysis could be utilized in relation to vaccine hesitancy. The following question is raised to achieve the latter:

In the last decade, social networks have become an important tool for opinion research [55] . Unlike other online sites, social media sites allow users from different countries to have public discussions about any topic, including vaccination, in real time [56] . In addition, public health professionals have been participating actively in these discussions, demonstrating that social media is not only a platform for real-time surveillance of vaccination hesitancy and infectious diseases but also a valuable communication tool for global health actors [8] . Sentiment analysis and opinion mining of texts have been gaining interest lately due to the increased availability of digital data and big datasets [57] . Mining the underlying emotions of comments, attitudes, and opinions through AI is a breakthrough that seems promising in identifying public opinions regarding vaccine hesitancy [58] . Sentiment analysis enables the categorization of opinions in correspondence to polarity (i.e., positive, negative, or neutral); emotions (i.e., anger and joy) or degree of agreement [58] . It is deemed as a categorization process, mainly at three levels: 1) document level that identifies overall opinion; 2) sentence level that detects attitude; and 3) aspect level, which is concerned with a specific opinion [59] . Monitoring vaccine hesitancy could include text mining to extract data from social media messages, analyzing it, classifying the stance towards vaccination [58] , and detecting the main subtopics of concern. It could also evaluate the intensity of agreement on the basis of a numerical rating scale [27] . With social media platforms rapidly turning into a free space, where people express opinions without prejudice, sentiment analysis serves as a powerful tool to evaluate vaccine hesitancy trends in the wider public opinion, with lower cost and larger coverage. Realizing this potential of sentiment analysis and its integration in analyzing vaccine hesitancy across the literature led to the conduct of this study, with the following question:

Given the growing number of studies employing sentiment analysis tool in vaccine hesitancy assessment and in this fast-evolving field, a thorough evaluation of the existing literature must be performed. In this study, a comprehensive systematic review was conducted, in which the literature was mapped; the main findings and methods used were summarized; and various aspects of the studies were evaluated, including their challenges, motivations, and recommendations, in addition to taxonomy analysis. These aspects were reviewed differently compared with most literature works. Most previous studies are concerned with discussing sentiment analysis from one scientific perspective, and no study has considered studying sentiment analysis with vaccine hesitancy with relation to various scientific domains. This review addressed this gap by not only studying and classifying J o u r n a l P r e -p r o o f its main findings based on computer science but also showing all the findings and discussing them from three perspectives; (1) social, (2) technological, and (3) medical.

A systematic review approach was adopted in this study to comprehensively evaluate the progress of sentiment analysis of the vaccine hesitancy approach worldwide. The search was conducted in accordance with preferred reporting items for systematic review and meta-analysis (PRISMA). Systematic review enables researchers to deeply understand a particular topic of interest and establish future insights [60] . It is also recognized for its structured method for research synthesis due to its methodological process and identification metrics in identifying relevant studies compared with conventional approaches [61] , thereby making it a valuable asset not only for researchers but also for students in post-graduate levels for developing an integrated platform of their research studies by identifying the existing research gaps and recent status of literature [62, 63] .

The literature search in this review covered research articles of peer-reviewed journals that are indexed in six renowned databases, namely, (1) Web of Science (Wos), (2) Scopus, (3) Pubmed, (4) IEEE Xplore, (5) Science Direct, and (6) Ovid. The search query was constructed by referring to the Medical Subject Headings (MeSH) terms, reviewing the literature, and consulting experts in the field. These databases were selected owing to their resilience and scientific soundness, and they were deemed sufficient and most suitable for this review.

The search was carried out in two stages: the first was conducted in February 2021 and upon the completion of the manuscript main findings; another round of related literature was carried out in July 2021 to ensure that more updated and recent literature was included. In all the used five databases, Boolean operators were utilized for the search, and three groups of key words (i.e., queries) were utilized in the process, as presented in Figure 1 and as the following:

 ("Sentiment Analysis" OR "Sentiment Classification" OR "Opinion Analysis" OR "Opinion Mining") AND (Vaccine OR Vaccination OR Immunisation OR Immunization) AND (Hesitancy OR Hesitant OR Refusal OR Denial OR Rejection OR Resistance OR Confidence OR Acceptance OR Reluctance OR Mistrust OR Distrust OR Misinformation).

Several inclusion and exclusion criteria were imposed while attempting to identify the most relevant articles in the study selection process. The date of publication was set from 2010 to July 31, 2021. In accordance with additional criteria, all papers, which included reviews, conferences, books, and research papers, were limited to those in the English language across all the selected databases. The remaining criteria were concerned with the inclusion of all the papers that discussed sentiment analysis and opinion mining and whose focus was on vaccine hesitancy.

First, article search resulted in the initial number (n = 3206), followed by removal of duplicated articles (n = 47). After title and abstract screening was conducted, a total of (n = 2,092) were excluded on the basis of the selection criteria. Sixty-seven articles were reviewed by full text and with 37 excluded. Thirty articles satisfied the inclusion criteria, and they were reviewed for data extraction, as shown in Table 1 .

J o u r n a l P r e -p r o o f 

Data were collected and extracted by seven researchers from each article in this review to analyze and draw the main aspects. The data collection items are presented in Table 1 . Every included article was thoroughly analyzed for various data that were predefined and agreed upon by all authors. The information extracted was general study characteristics, such as the study title and publication year. In addition, data related to methodological aspects were collected, including case discussion of vaccine hesitancy, the work applied, source data, the volume of data, and the duration of collection and analysis applied. Data related to challenges, motivations, and recommendations by authors were also extracted to inform prospective research on the difficulties and obstacles in the field, the advantages and suggestions for improvement in the future. A quality assessment was conducted on the basis of 11 criteria for all the included articles to ensure their contribution to the body of knowledge of the current review. The method used in this quality assessment was developed and utilized by [64] on the basis of the qualitative checklist of Critical Appraisal Skills Program and the accumulated list, as shown in Table 2 . This method was also utilized by [65] . This method ensures all the article parts are covered, from design to data acquisition, data analysis, and conclusion.

The criteria for quality scoring could be assessed as follows:

 A score of 1 if the criterion is fully met  A score of 0.5 if the criterion is partially met  A score of 0 if the criterion is not met

The score of each assessed article is between 0 and 11, in which 11 is the highest quality score. Therefore, an article with high quality score signifies high quality. The Appendix presents the quality assessment criteria.

As shown in Appendix, all the articles scored 7.5 and higher, which is an indication of making a valuable contribution of this review. However, most of the articles did not provide a comparison with previous studies. Further, many studies stated the bias of sample size as a limitation of their study.

Systematic reviews are review papers that discuss a topic of interest and identify their most remarkable findings on the basis of the author's perspectives. Many systematic reviews share a high similarity in terms of the approach they use for describing their protocol information. However, systematic review is an art, and that art could be conveyed upon the theme authors apply in their SLR papers to present most remarkable and interesting findings in the best manner they see fit. The main themes for the present systematic review are presented in Figure 2 . 

All the articles were classified to fall under three major categories and their subcategories. In the first category, disease-related sentiment to vaccine hesitancy was discussed. In the second category, discussion-based sentiment was discussed. In the last category, other sentiment and vaccine hesitancy related articles that do not fall in any of the previous categories were taken. The rationale behind this taxonomy design is owed to the nature of articles, J o u r n a l P r e -p r o o f as the main topic was around vaccine hesitancy and sentiment analysis, an observant literature pattern was observed and discussed between the authors, resulting in this final shape of the taxonomy categorization. In addition, creating the taxonomy in this manner enabled the categorizations of literature based on a common theme inspired by their most similar ideas and concepts, thus facilitating easier and more comprehensive understanding, as shown in Figure 3 . All the taxonomy analyses that contain the most important data, including the reference, data source, volume of data, duration of collection, discussed vaccine hesitancy and work applied, are presented in Table 3 .

This section discussed all the articles related to sentiment analysis and vaccine hesitancy from a disease perspective, i.e., vaccine hesitancy linked with (1) general disease, (2) specific disease, or (3) hybrid disease.

A total of (n = 1) discussed sentiment analysis regarding vaccine hesitancy generally. Rodríguez-González, Tuñas, Prieto Santamaría, Fernández Peces-Barba, Menasalvas Ruiz, Jaramillo, Cotarelo, Conejo Fernández, Arce and Gil [66] Discussed the importance of sentiment analysis in attracting scientific and industrial perspectives. The authors captured sentiments expressed across a set of tweets retrieved for a study about vaccines and general diseases during the period 2015-2018, with more than 1,028,742 tweets reviewed. A variety of different machinelearning techniques were used to correctly identified sentiment in tweets with the unbalanced data problem.

Specific Aside from those who discussed general disease, some authors mainly focused on sentiment analysis for specific diseases associated with incidents of vaccine hesitancy. The main topics of discussion were linked to major diseases, including (1) Measles, (2) Human Papillomavirus (HPV), and (3) COVID-19. For works discussed measles. The first work by Deiner, Fathy, Kim, Niemeyer, Ramirez, Ackley, Liu, Lietman and Porco [67] was on social media discussions, which they strongly believed may reflect public interest and stance regarding vaccine hesitancy for measles vaccine. A total of 58,078 Facebook (FB) posts and 82,993 tweets from January 4, 2009, to August 27, 2016, were analyzed for posting frequency and timing among individuals expressing vaccine hesitancy and those expressing opposing views. Machine learning and statistical analysis were utilized towards classifying social media posts as pro-vaccination, expressing vaccine hesitancy (broadly defined), uncertainty whether vaccine hesitant or not, or off topic. Their final results suggested an ongoing substantial presence of vaccine hesitancy and vaccination opposition in social media, in contrast with sporadic involvement by individuals favoring vaccination. Another disease for discussion was HPV. In the work of Zhang, Fan, Peng, Rao and Cong [68] , the authors argued that owing to the existence of a large amount of data for public sentiment analysis in social media, researchers could now study public opinions on HPV vaccines on social media by using machine learning-based approaches that could help understand the reasons behind low vaccine coverage. Therefore, they proposed three transfer-learning approaches to analyze the public sentiment on HPV vaccines on Twitter. They utilized fine-tuning bidirectional encoder representations from transformers (BERT) for 6,000 tweets, which were selected for annotation randomly between July 15, 2015, and August 17, 2015. The results suggested the efficiency of the proposed approach, which also assisted in helping find strategies to improve vaccine uptake. Loft, Pedersen, Jacobsen, Søborg and Bigaard [69] Other aspects, such as personal stories, were also found to be effective in creating positive dialogs on FB. However, the authors concluded that providing factual information to parents remains necessary and important to enable informed decision-making about HPV vaccination. Du, Xu, Song and Tao [70] Also conducted sentiment analysis on HPV, highlighting that understanding the leveraging causes for public concerns with HPV vaccination could support HPV vaccination promotion. The authors utilized ML algorithms for extracting the public sentiment on HPV vaccine between November 2, 2015, and March 28, 2016. All English tweets were collected from Twitter, with a total of 184,214 tweets. The results indicated a weak trend for "negative" tweets as opposed to "positive" tweets. Tweets that contained worries on the efficacy of HPV vaccines showed a relatively significant decreasing trend. This approach is believed to provide feedback to public health professionals to monitor online public response, examine the effectiveness of their HPV vaccination promotion strategies, and adjust their health promotion plans. Luo, Zimet and Shah [71] Developed an NLP framework to investigate the opinions on HPV vaccination reflected on Twitter over a 10-year period between 2008 and 2017. The NLP framework included sentiment analysis, entity analysis, and AI-based phrase association mining for a total of 287,100 tweets. The author results indicated that more negative tweets were identified in 2008-2011 and 2015-2016. Their work also aimed to identify the main topics reflected in negative and positive tweets and detailed tweet content. Their study could help public health researchers better understand the nature of social media influence on HPV vaccination attitudes and develop strategies to counter the proliferation of misinformation. Ortiz, Smith and Coyne-Beasley [72] Examined how social media may affect HPV vaccine uptake and HPV vaccine related awareness, knowledge, and attitudes via a systematic review. They analyzed a total of 44 relevant articles, and most of these works analyzed the valence, type, and frequency of social media content about HPV vaccination. The results found some associations that existed between potential exposure to negative anti-vaccine content and lower vaccination. The authors also concluded a lack of systematic and rigorous research examining the effects of social media and a need for further examination as social media increasingly becomes a source of health information. Aside from HPV, vaccine hesitancy associated with COVID-19 vaccination is a growing area of study. Mutanga and Abayomi [73] Discussed the COVID-19 pandemic in South Africa and raised how the success of such synergized efforts primarily depends on the people's attitudes and perceptions towards the multifaceted management of the pandemic. The authors applied the Latent Dirichlet Allocation (LDA) algorithm for the extraction of noteworthy topics, harvesting Twitter data between March 15, 2020, and April 30, 2020. Their aims were to discover what topical issues relating to the pandemic and what effects these issues have on compliance with regulations. Their results observed that alcohol sale and consumption, staying home, daily statistic tracing, police brutality, 5G, and vaccine conspiracy theories were among the topics discussed and around which attitudes and perceptions were formed by the citizens. The findings also revealed people's resistance to measures that affect their economic activities and their unwillingness to take tests or vaccines as a result of fake news and conspiracy theories. The second work by [33] discussed Tweet topics and sentiments relating to COVID-19 vaccination among Australians. The authors collected 31,100 English tweets between January and October 2020 from Australian Twitter users to understand such phenomena, and they analyzed tweets by visualizing high-frequency word clouds and correlations between word tokens. The authors also performed sentiment analysis to understand the overall sentiments and emotions related to COVID-19 vaccination in Australia. Their main aim was to use machine-learning methods to extract topics and sentiments relating to COVID-19 vaccination on Twitter. Their analysis identified attitudes towards COVID-19 and its vaccination, advocating infection control measures against COVID-19, and misconceptions and complaints about COVID-19 control. Their final results indicated that some Twitter users in Australia supported infection control measures against COVID-19 and refuted misinformation. The third work by Gbashi, Adebo, Doorsamy and Njobeh [74] discussed systematic delineation of media polarity on COVID-19 vaccines in Africa. The authors claimed that media communications could affect public perception and attitude towards medical treatment, vaccination, or subject matter. Thy collected 637 twitter posts and 569 Google News headlines or descriptions, retrieved between February 2, 2020, and May 5, 2020. Data were analyzed using three standard computational linguistic models. Their final results revealed that contrary to general perceptions, Google News headlines or snippets and Twitter posts within the stated period were generally passive or positive towards COVID-19 vaccines in Africa, and understanding these patterns in light of increasingly sustained efforts by various media and health actors was possible to ensure the availability of factual information about the pandemic. In another study by Jang, Rempel, Roth, Carenini and Janjua [75] , the COVID-19 discourse on Twitter in North America was tracked. The authors started data collection from January 28, 2020, until the end of May 2020, with a total of 319,524 tweets. COVID-19-related tweets were analyzed using topic modeling and aspect-based sentiment analysis (ABSA) and interpreted the results with public health experts. Their main aim was to investigate people's reactions and concerns about COVID-19 in North America, especially in Canada. Their final results discovered that topics and their trend are highly related to public health promotions and interventions. After training the data by using ABSA, 545 aspect terms were obtained, and 60 opinion terms were used for inference of sentiments of 20 key aspects selected by public health experts. The results also showed negative sentiments related to the overall outbreak, misinformation and Asians, and positive sentiments related to physical distancing. The fifth work by Praveen, Ittamalla and Deepak [76] Discussed the attitude of Indian citizens towards COVID-19 vaccine. The authors explained that as the process of vaccination was not made mandatory, skepticism across the public towards COVID-19 vaccines arose. A total of 73,760 tweets were collected for sentimental analysis to determine how the general perception of Indian citizens regarding COVID-19 vaccine changes over different months of the COVID-19 crises. The authors also performed topic modeling to understand the major issues concerning the general public regarding COVID-19 vaccine. Their final results indicated that 47% of social media posts discussing vaccines were in a neutral tone, and nearly 17% of the social media posts discussing COVID-19 vaccine were in a negative tone. Fear of health and allergic reactions towards the vaccine are the two prominent issues that concerned Indian citizens regarding COVID-19 vaccine. Hussain, Tahir, Hussain, Sheikh, Gogate, Dashtipour, Ali and Sheikh [56] Discussed the public attitudes on FB and Twitter towards COVID-19 vaccines in the United Kingdom and the United States. Over 300,000 social media posts related to COVID-19 vaccines were collected from March 1, 2020, to November 22, 2020. They used NLP and deep learning-based techniques to predict average sentiments, sentiment trends, and topics of discussion. Their main aim was to develop and apply an AI-based approach to analyze public sentiments on social media to better understand the public attitude and concerns regarding COVID-19 vaccines. Their final results found that the overall averaged positive, negative, and neutral sentiments were at 58%, 22%, and 17% in the United Kingdom, compared with 56%, 24%, and 18% in the United States, respectively. Public optimism over vaccine development, effectiveness, trials, and concerns over safety, economic viability, and corporation control were identified. Another work for COVID-19 [77] was an exploratory study, where the authors discussed the conspiracy theory among tweets from the COVID-19 infodemic. The authors began with a corpus of COVID-19 tweets (approximately 120 million) spanning from late January to early May 2020. Then, they filtered tweets by using regular expressions and used random forest classification models to identify tweets related to four conspiracy theories. Their aims were to use Twitter data to explore methods to characterize and classify four COVID-19 conspiracy theories and provide context for each of these conspiracy theories through the first 5 months of the pandemic. Their final results showed that misinformation tweets demonstrated more negative sentiment than non-misinformation tweets, and that theories evolve over time, incorporating details from unrelated conspiracy theories and real-world events. The work by [78] discussed the increasing use of social media as a source of health information, which contributed to vaccine hesitancy due to anti-vaccination content. A total of 75,797,822 tweets between January and August 2020 were reviewed to identify anti-vaccination tweets. The final results suggested that the BERT model achieved excellent performance, and it could be used to identify anti-vaccination tweets. The evidence of disorientation towards immunization on online social media has also been studied [79] . Authors collected a total of 318,371 tweets posted in 2018, mainly to propose a procedure for testing for the presence of short-and longer-term collective disorientation on Twitter. Their final results suggested that vaccine-relevant tweeters' interactions peaked in response to main political events. In addition, the smoothed time series of polarity proportions exhibited frequent large changes in the favorable proportion, superimposed to a clear up-and-down trend synchronized with the switch between governments in spring 2018, suggesting evidence of disorientation among the public.

In this second taxonomy theme, the vaccine hesitancy captured in studies by assessing online discussions and debates was presented. The articles in the latter discussed one of two points; (1) Online Misinformation and (2) Online Debates. All of these subcategories with their corresponding subsections and articles are discussed below.

At present, one of the greatest risks to human health comes from the deluge of misleading, conflicting, and manipulated information currently available online, including health misinformation. Vaccination is a topic particularly susceptible to online misinformation. Combating vaccine misinformation requires an understanding of the prevalence and types of arguments being made and the ability to track how these arguments change over time. A total of 11 studies fell under this subcategory. In the first work [80] , the authors explored the childhood anti-vaccine and pro-vaccine communities on Twitter. Tweets from influential users about childhood vaccines between July 1, 2018, and October 15, 2018, were assessed. A total of 139,433 tweets were collected, and 14,735 tweets with influence were identified and analyzed to discover the most discussed themes in each community. The results indicated a well-connected anti-vaccine community, where influential users widely share vaccine misinformation. Further sentiment analysis concluded that negative tweets populate pro-and anti-vaccine communities, thus confirming the popularity of negative sentiment on social media. In another study [81] , the authors adapted an existing typology to 1.8 million vaccine-relevant tweets compiled from 2014 to 2017 to understand vaccine misinformation. They utilized LDA topic modeling to extract 100 topics from the dataset, with the aim of adapting and extending the existing typology of vaccine misinformation to classify the major topics of discussion across the total vaccine discourse on Twitter. Monitoring a stance towards vaccination on Twitter messages has also been conducted in one study [82] , where the authors queried TwiNL for different key terms related to the topic of vaccination in a 5-year period, ranging from January 1, 2012, to February 8, 2017. The authors collected a total of 96,566 tweets from TwiNL. Their main aim was to develop a system that automatically classifies stance towards vaccination on Twitter messages, with a focus on messages with a negative stance. Their final result revealed that Support Vector Machines (SVMs) trained on a combination of strictly and laxly labeled data with a more fine-grained labeling yielded the best result, considerably outperforming the currently used sentiment analysis. Their results also showed that the recall of their system could be optimized at a slight loss of precision. Deiner, Fathy, Kim, Niemeyer, Ramirez, Ackley, Liu, Lietman and Porco [67] conducted J o u r n a l P r e -p r o o f a systematic reivew of 69 studies published between 2000 and March 2019 to identify main health misinformation topics and their prevalence on different social media platforms, focusing on methodological quality and the diverse solutions being implemented to address this public health concern. Their results stated that health misinformation about vaccines was very common (43%), with the HPV vaccine being the most affected. Health misinformation related to diets or pro-eating disorder arguments were moderate in comparison to the aforementioned topics (36%). Studies that focused on diseases also reported moderate misinformation rates (40%), especially in the case of cancer. Finally, the lowest levels of health misinformation were related to medical treatments (30%). A systematic scoping review [83] has also been conducted to summarize the methods used to monitor and analyze vaccinationrelated topics on different social media platforms, along with their effectiveness and limitations. A total of 86 articles in English, Spanish, and Italian were included for analysis. The final results showed that most studies focused on negative (n = 33) and positive (n = 31) sentiments towards vaccination, and they may have failed to capture the nuances and complexity of emotions around vaccination. Furthermore, 49 out of the 86 studies determined the reach of social media posts by looking at numbers of followers and engagement. Finally, [84] discussed vaccine hesitancy and illness perceptions. The authors collected responses from 338 participants between January 2015 and January 2017. Their aims were to determine the proportion of vaccine-hesitant parents, overall and by diagnosis type; identify parental beliefs about the causes of chronic medical or behavioral health conditions associated with vaccine hesitancy; and examine differences in these beliefs by diagnosis type. Their final results emphasized the need to address vaccine concerns among parents of children with ASD to reduce vaccine hesitancy in this population and prevent further subsequent vaccine hesitancy through the ongoing spread of misinformation.

Debates Eleven studies that assessed online debates in vaccine hesitancy could be further divided into (1) vaccine-hesitancy cases and (2) opinion. In one study [85] , the authors examined emergent communities and social bots within the polarized online vaccination debate on Twitter. They collected data by using a geosocial system from February 1, 2015, to March 9, 2015. In total, they collected 669,136 tweets published by 268,055 distinctive users. Their aim was to investigate the communication patterns of anti-and pro-vaccine users and the role of bots on Twitter by studying a retweet network related to MMR vaccine. Their final results discovered that pro-and anti-vaccine users retweeted predominantly from their own opinion group. They also found that bots displayed hyper-social tendencies by initiating retweets at higher frequencies with users within the same opinion group. In [86] , the authors used semantic network analysis of vaccine sentiment on online social media by constructing and analyzing semantic networks of vaccine information from highly shared websites of Twitter; They analyzed 26,389 tweets from April 16, 2015, to May 29, 2015. Their final results suggested that the semantic network of positive vaccine sentiment demonstrated greater cohesiveness in discourse than the larger, less-connected network of negative vaccine sentiment. The results also stated that the prevalence of negative vaccine sentiment was demonstrated through diverse messaging, framed around skepticism and distrust of government organizations that communicate scientific evidence supporting positive vaccine benefits. The third work by [87] adopted sentiment analysis of social media on childhood vaccination. In total, 40,359 posts on childhood vaccination were collected between January and December 2015. The sentiments were classified, and posts were analyzed using frequency, trend, logistic regression, and association rules. The authors developed a childhood vaccination ontology to serve as a framework for collecting and analyzing social data to use it for identifying concerns about and sentiments towards childhood vaccination from social data. Their final results suggested that childhood vaccination trends in sentiments were affected by news about vaccinations. Posts indicating parents' health belief, vaccination availability, and vaccination policy were associated with positive sentiments, whereas posts of experience of vaccine adverse events were associated with negative sentiments. The fourth work by [88] discussed whether vaccines for pregnant women is absurd or not and highlighted the nuance of language in social media posts about maternal vaccinations. Twitter, forums, blogs, and comments were used to extract data from 15 countries between November 1, 2018, and April 30, 2019. The authors used stance, discourse, and topic analysis to provide insights into the most frequent and weighted keywords, hashtags, and themes of conversation within and across countries. A total of 16,000 were included in the analyses. The main aim was to understand the predominant topics of discussion, stance, and associated language used on social media platforms relating to maternal vaccines. Other researchers discussed various online debates with respect to people opinions. The work by [89] presented Twitter as a sentinel tool to monitor public opinion on vaccination in Italy. A total of 180,620 vaccine-related tweets during the period September 2016-August 2017 were identified and collected to monitor the public opinion on vaccination through Twitter by using a machine-learning model to automatically assess opinion polarity. The final results found an increasing trend in the number of tweets on this topic. According to the overall analysis by category, 60% of tweets were classified as neutral, 23% against vaccination, and 17% in favor of vaccination. Vaccine-related events appeared to be able to influence the number and the opinion polarity of tweets. The other part of this subgroup included papers discussing sentiment analysis with relation to vaccine hesitancy for public opinions. The work by [90] discussed the adoption of text mining and sentiment analysis to analyze Italian YouTube videos concerning vaccination. The authors used co-occurrence network (CON) and sentiment analysis to analyze the topics of these videos from May 1 to October 1 for years 2017 and 2018. In 2017, 1898 videos were adopted for the analysis, while 822 videos were used for the analysis in 2018. The aims were to understand if and how the population's opinion has changed before the law and after the vaccination campaign by using the titles of the videos uploaded on YouTube. The CON confirmed that vaccinations were very disapproved before the law. However, after the communication campaign, people started to be less critical. The sentiment analysis showed that the intense vaccination campaign also promoted by medical doctors pushed the sentiment to change polarity from a prevailing negative opinion in 2017 to a positive one in 2018. The work by [58] discussed the vaccine hesitancy on social media. They used a hybrid approach to perform an opinion-mining analysis on 1,499,227 vaccine-related tweets published on Twitter from June 1, 2011, to April 30, 2019. Their algorithm classified 69.36% of the tweets as neutral, 21.78% as positive, and 8.86% as negative. The percentage of neutral tweets showed a decreasing tendency, while the proportion of positive and negative tweets increased over time. Peaks in positive tweets were observed every April. The proportion of positive tweets was significantly higher in the middle of the week, and it decreased during weekends. Negative tweets followed the opposite pattern. Among users with ≥ two tweets, 91.83% had a homogeneous polarized discourse. Opinion mining is potentially useful to monitor online vaccine-related concerns and adapt vaccine promotion strategies accordingly. The work by [91] discussed how Twitter users react to TV broadcasts dedicated to vaccines in Italy. For understanding of such phenomenon, the authors downloaded 12,180 tweets pertinent to vaccines, published by 5447 users, and 276 users tweeted during both broadcasts. Quantitative and qualitative methods were used to analyze the sentiment of vaccinerelated tweets. The main aim was to analyze the use of Twitter during these broadcasts dedicated to vaccines and explore the potential of this kind of media monitoring for informing public health practice. The final results suggested that sentiment was positive in 50.4% of tweets, negative in 37.7%, and neutral in 10.1%. The work by [92] discussed the media news on vaccines and vaccination in India. The authors analyzed 1788 news on immunization and vaccines published in English during November 2015-May 2020. Their main aim was to study the profile and vaccine sentiments of the online media news in India. Their final results suggested that the news focused on immunization program in 59.1% and vaccine hesitancy in 7.7% items. In addition, the negative sentiments focused on adverse events, vaccine hesitancy, and resistance. The news volume and negative sentiments were largely linked to the measles-rubella vaccination campaign phases in India. Another work [93] discussed about the sentiment analysis of social media data in vaccination. Data were collected from November 23, 2019, to May 15, 2020. The collected tweets were 100,000. The data were then manipulated in spreadsheets by using another library called Panda, where each data was divided into separate fields. The SVM classifier was used to classify the sentiment. The main aim was to provide solution on the sentiment analysis of approximately 100,000 tweets accumulated using Textblob and SVM classifier. The final results indicated that using SVM algorithm could provide a slight difference and a better performance to study the opinions of people in vaccination. In the last work [94] , the challenges and opportunities for harnessing social media in the modelling of pandemics were discussed. Social media data from March 22, 2020, to July 20, 2020, were included in compartmental models to understand such phenomenon. The main aim was to examine the challenges and opportunities inherent in the use of social media data in infectious disease modelling, with a particular focus on their inclusion in compartmental models. The results suggested that the interactive, immersive nature of social media may reveal emergent behavior that does not occur in engagement with traditional mass media or conventional surveys. 

This section aimed to discuss some of the most important elements in this review, including the issues and challenges faced by researchers, followed by motivations of research in this area. The elements of recommendations was highlighted to show the message of previous researchers to their future peers. This topic discussed important remarks, not only from computer science and technology perspective, but also discussed various significant findings that could be linked with social, medical, and health science. Therefore, to give this topic a fair presentation, all the major points, namely, Challenges, Motivations, and Recommendations, were discussed from three perspectives: (1) Technology, (2) Social, and (3) Medical. The discussion was agreed upon and drafted by authors who come from different scientific backgrounds (Social, Technological, and Medical) to present the topic in the best manner possible.

The sentiment analysis for vaccine hesitancy has three key challenges: technology, social, and medical. Technological challenges relate to those faced by computer and data scientists, while social challenges are faced in everyday scenarios, which could circle around normal people and their behaviors. Medical challenges relate to health and medicine. All of these main classes of challenges with their corresponding sub-challenges are presented in Figure 4 and discussed below. Technology As previously discussed, technological challenges are mostly faced by computer and data scientists working with data. In that regard, data were unsurprisingly the main aspect of discussion, with their various subtypes such as nature, collection, annotation, and processing.

Data Nature Nature challenges refer to issues that make understanding the data or even analyzing them difficult; when looking at the trends of challenges in this regard, they could be attributed to the vagueness of natural language [89] , which could include ambiguity of data [89] , especially in cases where slang language [86] , sarcasm and irony [58] are used. In addition, other aspects in this class could be attributed to the high level of data noise [66, 85] , including spelling mistakes and the use of non-standard text [73] . Finally, complex views on the same topic [89] , a variety of online content [90] , objectivity and subjectivity [89] , the structure of data [87] , or even their uneven distribution could pose significant challenges in determining the nature of the data [68] .

When working in the area of sentiment analysis in any topic, data collection is an integral part, especially because data are provided from social media outlets. Nevertheless, collecting such data does not come without issues. The issues found were linked to (1) availability of data, (2) volume of data collected, (3) location where data were extracted, (4) language of the collected data, and (5) duration for collecting the data. For data availability, the issues discussed were the consistent change in online content [95] , which could be deleted or become private [58] ; the variability of content [86] ; data access restrictions [88] , especially for data that could only be obtained from public domain [92] ; and the strict character limits [81] . After that comes issues in the volume of data collection, including difficulty to construct large dataset [68] and the issues of small tweet size that may play a role in influencing the analysis [70] . As for those who were challenged by the location of data collection, their issues were concerned with collecting from single location [58] and not including all geographical areas [96] or online communities [96] . Another aspect of concern was data collection from language perspective, where its main issues discussed language bias [58] , especially for English tweets [58, 88, 92] . For researchers who were stressed out on the duration of data collection, their main issues included focusing on a limited period of time [80] , short period of collection [87] , and changes within the timeframe of data collection [85] . Other rare challenges in collecting data were in relation to the cost of collecting the data [68] and ethical concerns, given than these data involve human subjects [87] .

In sentiment analysis, after data are collected, they need to be pre-processed, and part of this process includes labeling the tweets (Data) to understand their meaning. Challenges in such context includes the complexity of the annotation [86, 88] , and such thing could be attributed to different reasons, including cost [68] and lack of annotation [68] due to lack of experts to assist [66] . In addition, other annotation issues included manual annotation for a high number of posts [88] , thus making researchers suffer while conducting it [92] , and many in academia consider it as a laborious task [82] . Annotation could also be a demanding process because it requires high levels of consistency [86] and precision [91] , with high levels of subjectivity [88] and ambiguity [86] .

Data Processing After the collection of data and their annotation, processing the data in sentiment analysis is considered a challenging topic [89] , because it is conducted for various purposes. However, these processing purposes certainly come with challenges, mainly related to (1) Techniques, (2) Analysis, (3) and Monitoring. For issues included in the processing techniques, they discussed the difficulty in applying intelligent techniques, such as Deep Learning [68] , or even the processing time taken by traditional processing techniques, such as ML [68] . Additional issues included the consistent pursuit to improve machine learning algorithms [58] ; providing better machine classification than humans classification [67] ; the requirement of special skills to enable the processing techniques, such as features engineering [68] , and the existing techniques providing very close results [66] . Aside from techniques, analysis challenges were aimed towards different aspects, such as tools. These researchers discussed the existence of insufficient [85] and inaccurate tools [66] , the need for more tools [66] , and that the current sentiment analysis should not be applied as an independent tool and must be a component of broader strategies [58] . Other analysis issues involved analyzing the sentiment, including the challenge in identifying the sentiment expressed in a piece of textual information [66] , which could be attributed to the presence of emojis [93] , bots [85, 91] , hashtags [58] , links [58, 91] , positive and negative emotions [87, 95] , abbreviations [73] , and missingness [95] . Moreover, other sentiment analysis challenges were related to social factors [91] , such as community differences in different timeframes [96] ; the existence of misleading information [81] ; nonrepresentative data [69] , where same words may have different meanings in the text [68] ; and different topics of discussion [67, 87, 88] . Other analysis sub-challenges included the required efforts in dealing with some type of news, such as fake ones [90] ; the high level of analysis precision [91] ; and the time consumed in the process [90] .

As for last class of data processing sub-challenges, their main aspect of discussion was utilizing the processing capabilities for monitoring purposes. Across the main issues discussed was the fact that monitoring social media data requires skills to analyze them, and that in itself could be challenging [89] .

Social Another class of the main challenges in this review refer to social aspect challenges. This type often discuss challenges mostly linked to social aspect, including people, public, users, and many individuals who form the main circle of interest for social science aspects. Different sub-challenges are included and discussed in this section, including, Understanding, Emotions, Beliefs, Behavior, and Strategies.

Understanding Understanding sentiment analysis components is essential to design and understand the data that reflect people views. Meanwhile, certain aspects were found in the literature with respect to (1) the Nature of available online content, (2) the Variation of available online content, and (3) the Significance of available online content. Challenges with respect to the nature of content aspect discussed issues, such as difficulty to understand [66, 86] , which may be attributed to content ambiguity [68, 89] , misconceptions [69] , rumors [92] , misidentified [71] , and poor content [86] . Other scholars stressed on challenges linked to the variation of content due to differences in social media content [95] , various factors that may have an effect over the sentiment [91] , and the online content always changing [95] . The hast sub-challenge group in the aspect of understanding discussed the significance of available online content with respect to its debate, which may be a topic with global importance [88] , not to mention these topics online reflect a broad spectrum for special population and not the general ones [86] , and the basic fact that many misinformation are available online [81, 95] .

Emotions have always been a significant factor to understand and know people. Thus, they come with a fair share of challenges with respect to the sentiment analysis with vaccine hesitancy. Previous literature showed that the most significance emotions were associated with causing a sense of fear [67] ; a great deal of confusion [80] ; and distrust among some individuals, which may lead to a decrease in vaccination rates [67] .

Beliefs People's beliefs could be attributed or linked to their emotions, where the former may affect the latter or vice versa. Discussing them separately could enhance the understanding. In this study, different beliefs of people were amongst the most important challenges faced from social aspect, and majority of these beliefs were attributed to either (1) Personal Beliefs or (2) Harmful Facts. Personal beliefs discussed religion [85] , the existence of conspiracy theory [85, 95] , and the violation of personal freedom of choice [85] . However, some belief challenges were based on harmful facts, including the existence of many harmful information online [87] , which may be presented with facts, mimicking the language of mainstream news [81] . As for last issue in this sub-class, it discussed that some arguments against topics, such as vaccine, were based on well-known research published in the past and have been proven to be wrong [85] , and due to the existence of social bots that could influence opinion trends [85] .

Behavior Behavior is the result of emotions and beliefs, and it holds significance over the sentiment, as people's behavior and attitudes are reflected in their tweets and posts. With respect to sentiment analysis of vaccine hesitancy posts, the behavior presence comes with its issues that include the behavior of people (1) online or (2) offline. In the online aspect, if people knew their tweets are being analyzed, they may stop posting [87] , or when people are communicating online, trying to focus on correcting their misperceptions may have adverse effect even if the intention is the opposite [69] and pose a significantly negative effect on vaccine-related behavior [95] . For the behavior of people outside the umbrella of social media, some of the issues discussed included people's refusal to get vaccinated in certain areas [67] , which turns negative public debate into a rapid decline in vaccination coverage [69] .

Strategies Strategies are defined in the context of this review as any means or efforts taken for the benefits of the public for any particular purpose. They are not exclusive with social science; rather, they may be found with respect to other domains of science, such as the ones in this review or even others. In the area of sentiment analysis and vaccine hesitancy from social perspective, the challenges linked to strategies were discussed with relation to their (1) Coverage, (2) Support, (3) Communications, (4) Observations, (5) Promotion, (6) Targeting, and (7) Monitoring. The main issues in strategy coverage included insufficient coverage [86] , followed by those related to strategy support, including the users needing to tweet in support of vaccine hashtags [81] . As for sub-challenges associated with communication strategies, the main issues included the existence of poor communications, which could be a cause for concern [90] and have an adverse effect on vaccine acceptance [86] . For observations, their main issue of concern was in relation to observing tweets over longer periods of time [85] . As for promotion strategies, the issues faced were linked to the vagueness of the current promotion strategies and their lack of information and persuasive power [85] . The next group was basically more concerned with issues regarding targeting strategies, which should focus on dealing with fake news [90] ; the lack of focus on individual's reason for not being vaccinated [86] ; the inability to ascertain the identity of the user; and the information source credibility, which could be more important for users to gauge validity [81] . Monitoring strategies was the last sub-group discussing issues and challenges in this social aspect. The main issues reported in the literature were concerned with the challenge of monitoring [89] , the significant difference in the idea to know monitoring strategies, and whether these strategies have different effects on viewers' behavior [95] . Other monitoring strategies issues reported focusing on areas that have not yet been introduced before [88] , such as community differences during different timeframes [96] and not being able to monitor the general population who do not use Twitter [58, 89] , which could introduce population bias [70] . Other monitoring strategies reported the issue of monitoring influencers and users from each community, which may have different types [96] , not comparing across different times [96] , and only focusing on a specific period of time [96] .

The last challenges in the main class discussed medical health challenges. These challenges are defined as any sort of issues that may have an effect on health or relates to medical aspect in any way. As this topic of review is part of a larger medical case related to vaccine hesitancy, the challenges in this regard were best described with respect to medical from different sub challenges, including Understanding, Support, and Treatment.

Understanding Medical understanding on these sub-challenges relates to how medicine was challenged to understand various sides related to vaccine hesitancy. The issues in this class discussed (1) Disease and (2) People. The first group of issues discussed the existence of poor medical understanding on vaccine hesitancy [86] , which, despite the huge efforts to address it, is still prominent in developing countries [85] . As for understanding people, the main issues were concerned with medically understanding the rationale and causes behind their attitudes [86] , which could introduce refusal for vaccination [67] and constant changes over time [86] .

Medical support is the second subcategory in this main class; it discussed various medicine efforts to assist against vaccine hesitancy. The majority of issues in this regard falls into (1) Analysis and (2) Strategies. The first one discussed issues, such as the difficulty to analyze a high volume of misinformation related to vaccination [81] , the existence of low-quality data [90] , and the lack of benchmark or baseline data to compare with new research [91] or even across different times [96] and geographical locations [96] . The second one discussed issues from other perspectives, including the inability to explore possible variations for different vaccine products [89] , not relating historical events related to vaccination to patterns of opinions reflected on Twitter over time [71] , or the difficulty to draw conclusions [69] .

The last sub-challenge class relates to issues found from a treatment point of view. All the challenges in this class were found to be significant and important, and they could be associated with four different entities: (1) Human, (2) News and Social Media, (3) Government, and (4) Research. The first discussed issues, such as the consistent increase in vaccine refusal [85] , which was identified as a global public health challenge [92] , especially for populations raising doubts on their safety [80] . As for the issues discussing News and Social Media, the main challenges were attributed to the increasing number of harmful posts [87] that provide a large number of misinformation online [81, 90, 92, 95] . These misinformation could lead to confusion [80] and misconceptions [69] , eventually leading to a challenge for vaccination uptakes [85] . As for government, the main challenges were associated with the negative public opinion that affects vaccination coverage [69] , leading to the issues of vaccine hesitancy [86] and lower vaccination rates than expected [71] . Other issues were associated with governmental efforts due to their poor recommendations [71] and vague strategies [85] , which resulted in few adoption of vaccine program [71] . The treatment issues were concerned with the role of medical research in vaccine hesitancy due to limited [88] and lack of research [72, 87] , along with the sole existence of specific research in underlying vaccine hesitancy [86] .

Motivations are also referred to as significances that make researchers and academicians drawn to any domain of research. They show the main benefits of pursuing a particular area of research and exploring its potential benefits.

When it comes to this review for sentiment analysis and vaccine hesitancy, the motivations are also not considerably different from the challenges in how they are presented with respect to technology, social, and medicine. Motivations associated with technology present significance that could be seen from computer and technological perspectives. As for social, the same thing could be applied, where the social aspect is the main drive in presenting its motivations. Medicine is no different, and its motivations are presented from medical perspective and linked to health to facilitate broader understanding of such potentials ( Figure 5 ).

As previously discussed, technology motivations benefit researchers from technological perspective, and they could be attributed to three subtypes: (1) Availability of Data, (2) Usability of Data, and (3) Data Accessibility.

Data Availability Data availability refers to the significance of having plenty of available data to analyze, one of the best things a computer researcher could work with. The reason is because if no data exist, no technology and computer research could exist. The motivations identified in this regard are that sentiment analysis is a wide area of research and its availability of data is large [66] [67] [68] , especially in health-related topics [90] . In addition, owing to the availability of a huge number of data, researchers could work conveniently in the NLP field [71] to analyze millions of messages [81] and large-scale information [85] .

Data Accessibility Data accessibility could be a huge benefit for researchers in the sentiment analysis field, and motivations in this regard could be attributed to the ease of accessing information [90] , the possibility for collecting data in many languages [93] , and the fast accessibility of information [89, 90] , not only for specialists but also for the public [67, 87, 88] , as it could be conducted in real time [58, 86] .

The usability of data could be understood from what sort of various uses it could be aimed at. When scanning the motivation in this regard, they were found to be associated with different purposes, including tracking [67] ; analysis, especially for deep learning [68] ; NLP [71] ; detection [95] ; and text mining [58] for user data [67] and even smaller communities [85] .

Social This section addresses the motivations found by previous researchers in this field with respect to the social aspect. These motivations could be linked to three directions; (1) User, (2) Government, and (3) Websites.

User User motivations could be identified as the benefits presented to users of social media websites used in the text mining and sentiment analysis, because they present users with a method to freely share their opinions [71, 90] , which show real reflection [67] , and share them fast [68, 90] . Aside from opinion sharing for users, some of the motivations in this category are significant because they allow studying different types of users and their opinions, including the public [68] or more specified type of people, including those who are influential [96] , from different backgrounds [93] , from different genders [95] , or even celebrities, influencers, politicians, and country presidents [93] .

Government Social motivations for governments are slightly different from those related to users. User's benefits come in a small point of view, but governmental motivations covers a larger scale, thus showing significance and importance. The motivations in this sub-category could be discussed from four beneficial ideas; (1) Surveillance, (2) Assessment, (3) Communications, and (4) Information sharing. For surveillance motivations, they include using it as a method to monitor diseases [67] and other medical concerns [58] . Such method could be used with public health institutions [89] , and it is different in comparison with traditional media [91] . Other uses and benefits included using it as a means for safety monitoring [88] , because this kind of surveillance of real-time Twitter information flow could provide timely updates [70] and enable the monitoring of special communities, such as the anti-vaccine ones [85] , or even the content shared by social media users [81, 92] . The second-motivation subclass for governments included the assessment aspect, which enabled governments to assess different entities, including public discussion [67] and opinion [70, 89] with the aim of understanding the negative sentiment for skepticism and distrust of government organizations, especially in the case of scientific evidence supporting positive vaccine benefits [86] . The next is government communications, where governments benefit from using sentiment analysis for vaccine hesitancy with respect to enhancing people's beliefs [86] , which could contribute in influencing people's decision to accept, delay, or refuse vaccination [58, 90] , shaping the opinion of the public [92] , and understanding their level of fear and anger [95] to promote vaccination [70] . The other motivations in this class could benefit governments in redirecting their efforts [73] to help with their public policies [88] and aid in developing proper interventions [88] , strategies for health communication [85] , and surveillance [92] . The last governmental motivations included some of the potentials presented by sentiment analysis in various information sharing, including real-time information [58, 70, 85] , which could aid in developing successful efforts to fight misinformation [81] and reaching to the public [80, 92] , very important for decision makers [69] .

Websites Website motivations relate to all significances that reflect the social media outlets utilized in text mining and sentiment analysis. Most of the identified motivations in this regard are linked with the (1) Capabilities of Information from these social outlets and their (2) Analysis Capabilities. The capability motivations included how largely the information could be shared [67] at low cost [89] to enable proper sharing of emotion [68, 95] , opinions [85, 89] , and attitudes [86] . As for the analysis capability motivations, they are related to various aspects, including studying online behavior and main social media topics [71, 81] and their change over time [81] . Other analysis capabilities could assist researchers [71] , public health professionals [58, 85] , and decision-making bodies [70] .

Medical Medical motivations in the presence of sentiment analysis with vaccine hesitancy resulted in a number of motivations. All of these motivations show the potentials of this area with respect to health perspective in various aspects, including, (1) Treatment, (2) Monitoring, (3) Management, (4) Strategies and (5) Research.

Treatment Medical motivations start with benefits that could be associated with treatment of people. In that context, two areas of motivations were mostly discussed: (1) Disease Prevention and (2) Treatment Improvement. For the first, across the benefits identified was the role of health information online in the prevention of disease [68, 95] , eliminations of viruses [67] , handling of outbreaks [73] , preventing millions of deaths [67] , and other health issues [66] , not to mention the management of healthy and sick people [73] . Public health could also be improved [86] to establish preventive measures [71] . Other motivations were more concerned with treatment improvement [95] ; maintaining rates of treatments [87] ; and promoting the safety of some treatments, such as vaccination [58, 90] , health messages [95] , and other health strategies [66] . Other treatment improvement motivations were attributed to improving treatment coverage, especially vaccine [71, 86] , which in turn could assist in allowing more responses to health-emerging concerns [89] to assist authorities [90] , health professionals, and agencies [70] to share their experiences [68] and provide useful implications for health campaigns [96] , health initiatives [91] , and health strategies and guidelines [68, 80, 85, 88] .

The scientific motivations for researchers' works in relation to sentiment analysis present different significances that contribute to research and scientific work. A main author's motivations in that capacity were associated with conducting research, which enables studying different societies [93] , public opinions [68] , public discussions, [58] and public concerns [71, 87] . Other medical research motivations were concerned with understanding health impacts [86] and perceptions [88] in various topics, including health tweets and topics [69, 95] , negative tweets [71] , and how people exploit social network for health purposes [90] .

The last part of the discussion in this review is recommendations, known as the suggestions and advice given to new researchers. These recommendations are simply the message of previous researchers to their future peers who may be other researchers. Recommendations could also be directed to other than academicians, such as governments, public, and individuals, depending on the nature of the study. In the context of sentiment analysis with vaccine hesitancy, the same aspects of discussion could be followed with regard to presenting the recommendations with respect to Technology, Social, and Medicine. In the context of technology recommendations, information technology is addressed in any aspect that relates to science. Recommendations meant for social science discussed aspects that reach out to public and people. As for recommendations associated with medicine, any sort of medical related suggestions or advice are addressed from that context. 

The technology recommendations associated with analysis enhancements were concerned with three points: Features, Techniques, and Datasets. These points were deemed most significant in technology, and pointing them out could assist researchers in future analysis for sentiment analysis while generating more knowledge from the analysis topics they target. For the first aspect that relates to features, most of the recommendations were associated with considering emojis [58, 85] and adding them as part of the sentiment analysis [71] , including integrating them with machine-learning models [85] . Other feature recommendations were associated with considering tweets associated images and URL [58] and the number of retweets [71] . Other recommendations were concerned with enhancing analysis techniques, and the literature suggestions in this regard included developing more accurate algorithms for sentiment analysis [71, 90] , trying different ones for topic extraction [73] , emotion classification [87] , and improving the results [93] . Further recommendations also discussed using machine-learning approaches [58, 82, 88] and assessing their classification [66] . The last analysis enhancement recommendations were associated with datasets. The literature in this aspect discussed that larger datasets should be used for the analysis [67] and training of the models [82] . Others also suggested that datasets with more labeled data should be used [82] in addition to combining more data sources [87] and using social media platforms other than Twitter, such as Facebook, blogs, and online message boards [85] .

Research Topics For the recommendations associated with research topics, they show what previous literature in the technology field encourage in the future with respect to new or existing research topics. The recommendations in this context were concerned with studying positive and negative sentiments in vaccination-related topics [58] and more studying categories [71] . In addition, other recommendations included analyzing he tweet history of users [85] , extracting sentiments and emotions from the harvested tweets [73] , and including more timeframes [85] and contents [95] .

This section addresses the recommendations suggested by previous researchers in terms of social aspect. These recommendations could be linked to three directions: (1) Vaccine, (2) Public, and (3) Communications.

Vaccine Vaccine-related recommendations from the social aspect were linked to different aspects. The suggestions included the encouragement of more positive approaches and strategies towards vaccine [67, 87] , designated research towards understanding people's attitudes and behavior towards vaccine [88, 95] , and understanding the effect of posts in relation to vaccine uptake [88] .

The public recommendation in the social aspect included points associated with the importance to publicize policies on free vaccinations [87] ; exploring and assessing public sentiment [68, 70] , especially for influential users [85, 96] ; and how sentiment affects the Twitter follower network [80] . Other recommendations discussed the importance of investigating users who spread misinformation [96] , studying specific causes underlying vaccine hesitancy [71, 86] , using social media in informing the public [67, 90] , and communicating with their users [81] . This approach could provide a suitable means for tracking the evolution of communities to significant events [87] .

The last point of discussion discussed the recommendations found in previous research works in relation to sentiment analysis and vaccine hesitancy. The recommendations are either meant for (1) Health Monitoring or (2) Health Improvement.

Health Improvements For health improvements, several recommendations were found on the basis of research works identified. Amongst the most important points discussed was the need to improve public health communication to vaccine hesitancy [86, 90] and promote an efficient plan to resolve public concerns to increase vaccine uptake [70] . In addition, public health professionals could produce more emotionally appealing online content for vaccine [85] and provide clarifications on topics and issues that may stop people from being vaccinated [71] .

Health Monitoring For health monitoring from a medical aspect, the recommendations were associated with the idea of using social media monitoring to design strategies for vaccine [58] and continually monitoring any vaccine-related topics of discussion, especially arguments and rumors [87] , to improve vaccine confidence and vaccination coverage [86] .

This study provides an up-to-date overview of the state of sentiment analysis employment in analyzing vaccine hesitancy, a phenomenon that is considered a major threat to global health and that has witnessed a dramatic escalation in recent years due to varying factors, of which social media interaction has been playing a major role. This review is clearly a multi-perspective study, where the focus was not only on sentiment analysis, a computer science aspect, but also on integrating it equally with two no less important aspects: social and medical. Therefore, implications must be drawn considering that in mind.

Technology and Computer Science Discipline: this area, in spite of its usefulness and capability, still requires more efforts with regard to data and processing as follows:

 Not all social media data are easily accessible for researchers and scientists and as a result, future comers could find it difficult to work in the area at the beginning. Therefore, SM should make their data more accessible.  Not all social media outlets could provide the options to crawl their data, thus hindering future research in the topic. Therefore, more SM platforms should integrate information-crawling APIs for their public user data to be used for research purposes.  Processing information requires far more research efforts and technical interventions across different domains and using more sophisticated techniques.  Most of the studies focused on the English language due the availability of pre-processing tools, datasets, dictionaries, and labeled data. This implication resulted in a lack of understanding on vaccine hesitancy from the perspective of sentiment analysis.  Data labeling is another challenge faced by data analysts due to the large volume of data, where millions of records required labeling before the classification task.

Social Science: this review confirmed that most of the misinformation about vaccine hesitancy is received via social media. Therefore, understanding sentiment analysis components is essential to design and understand data that reflects people's views. Thus, paying attention to individuals' emotions, such as fear, confusion, and distrust towards vaccine uptake, is critical.

 A consensus from the previous studies reviewed was that spread misinformation online is one of the main factors towards vaccine hesitancy. Therefore, in the present review, stakeholders are encouraged to explore and assess public sentiment, especially influential users. Moreover, users who spread misinformation among people must be investigated.  This review confirmed that people's confidence in healthcare workers have a high influence on their vaccine decision. Consequently, trust must be built by creating a direct hotline between them to revise their questions and concerns about the vaccine.  This review showed that lay and expert sources could communicate with people to encourage them to uptake vaccine effectively. However, ordinary people may be comparatively more effective on other people in terms of vaccine uptake.  This review advanced several factors that may be useful for developing a measurement to assess people's vaccine hesitancy.  This review proved that social media is an outlet to receive and send influential messages to the public. Therefore, encouraging celebrities, influencers, politicians, and even country presidents to be agents is necessary to raise awareness for vaccine uptake via social media.  This study confirmed through reviewing studies that people follow religious men for life matters. Therefore, the religious men and houses of worship should have a responsible role in promoting people to receive the vaccine.

Medical/Public health: this study provides an overview on public health and the medical standpoints of the issue and presents recommendations to policymakers and healthcare workers on the basis of the rigorous examination of studies regarding the applicability of sentiment analysis tool and its potential to aid in the understanding of vaccination hesitancy determinants. The study strongly recommends that sentiment analysis be adopted by governments and institutions that are concerned with public health issues and be utilized to reach the huge proportion of public opinions, allowing for real-time examination of their thoughts and trust levels and apprehensions in vaccines to devise more effective policies and communication methods. However, policy makers and public health personnel should keep in mind that this tool is still developing, and that it has limitations that need to be addressed as follows:

 A major limitation is the inefficiency of sentiment analysis to catch the distinctions and intricacy of emotions and opinions, with sarcasm as an example. In addition, the misuse of bots to convey anti-vaccine messages or retweets at a high rate may potentially skew conclusions to overestimate the phenomenon or underestimate the effectiveness of the implemented strategies.  The current expansion of social media is not evenly spread across countries; this may result in overrepresentation in terms of vaccine hesitancy detection of particular regions, specifically in analyses that do not take geography into account.  The inaccessibility of underprivileged regions to social media could definitely impede the utility of sentiment analysis in assessing vaccine hesitancy.

 Another problematic issue for policy makers and public health entities is the ability to respond quickly based on tracked trends via sentiment analysis to public opinions that are shifting at a very fast pace. Therefore, collective efforts by all parties and agencies should urgently set up plans directed to address this critical issue to initiate timely and effective measures.  Sentiment analysis could be used as a part of wider strategies and in conjunction with surveys and other traditional approaches of gauging community perspectives, with the hopes that sentiment analysis approach could advance in the near future to tackle these issues, taking into account the continuous evolution of the field.

This review confirmed that the anti-vaccination movement is gaining momentum and influencing more people through the Internet and social media, thus making the dissemination of false facts and statistics easier. In this research, all the articles over an 11-year span (from January 1, 2010, to July 30, 2021) that discussed vaccine hesitancy in relation to sentiment analysis were systematically reviewed. The research motivation for this work was the availability of many vaccinations for COVID-19, which presented many false information and rumors about each of them and their effects. The phenomenon of vaccine hesitancy continues to undermine the countless efforts all the world governments are desperately taking to encourage their citizens to become vaccinated. More efforts should be directed towards social media outlets and the misinformation presented in them, which could encourage many to become vaccinated and by default fight the current pandemic. This systematic review was an attempt to cover sentiment analysis with relation to vaccine hesitancy and discuss the most important literature remarks from three aspects: technology, social, and medical. It also addressed the main highlights: the protocol that explains how the last set of articles was chosen and a taxonomy analysis of current papers in the field and previous research efforts in the form of challenges, motivations, and recommendations. To the authors' knowledge, this review was the first to discuss the main paper's highlights with relation to factors together in one work (technology, social, and medical). Further research efforts are warranted in this area, not only from one discipline but from many scientific discipline integration and collaboration. Those who lost their lives owing to pandemics and diseases are far more than those who lost their lives due to vaccine complications, and everyone is strongly encouraged to become vaccinated and return to their normal lives. Individuals and scientists from all over the world stand together in stopping vaccine hesitancy and encouraging people to being vaccinated.

The authors declare no conflict of interest.

The race to a COVID-19 vaccine: Opportunities and challenges in development and distribution

Analysis of the COVID-19 Vaccine Development Process: an Exploratory Study of Accelerating Factors and Innovative Environments

First month of COVID-19 vaccine safety monitoring-United States

Vaccine confidence in the time of COVID-19

Understanding vaccine hesitancy around vaccines and vaccination from a global perspective: a systematic review of published literature

Vaccine hesitancy, vaccine refusal and the anti-vaccine movement: influence, impact and implications

Vaccine hesitancy: not a new phenomenon, but a new threat

Anti/Vax: Reframing the Vaccination Controversy

Anti-Vaxxers' and Vaccine Citizenship: Insights for Medical Education

Communicating science-based messages on vaccines

Addressing mistrust about COVID-19 vaccines among patients of color

The benefit of the doubt or doubts over benefits? A systematic literature review of perceived risks of vaccines in European populations

Attitudes to vaccination: a critical review

Mapping global trends in vaccine confidence and investigating barriers to vaccine uptake: a large-scale retrospective temporal modelling study

Time to remember: Vaccines don't cause autism, The Brown University Child and Adolescent Behavior Letter

Looking beyond the Decade of Vaccines

The mediating role of health literacy on the relationship between health care system distrust and vaccine hesitancy during COVID-19 pandemic

COVID-19 related medical mistrust, health impacts, and potential vaccine hesitancy among Black Americans living with HIV

HPV vaccination in a context of public mistrust and uncertainty: a systematic literature review of determinants of HPV vaccine hesitancy in Europe

COVID-19 Vaccine Hesitancy: A Community-Based Research in Turkey

Socioeconomic Determinants in Vaccine Hesitancy and Vaccine Refusal in Italy, Vaccines

Vaccine Refusal and Measles Outbreaks in the US

Resurgence of Measles in Europe: A Systematic Review on Parental Attitudes and Beliefs of Measles Vaccine

Understanding confidence in the human papillomavirus vaccine in Japan: a web-based survey of mothers, female adolescents, and healthcare professionals

Polio vaccination in Pakistan

Understanding vaccine hesitancy in polio eradication in northern Nigeria

Drivers of vaccine decision-making in South Africa: A discrete choice experiment

COVID-19 zugzwang: potential public health moves towards population (herd) immunity

The Salzburg statement on vaccination acceptance

WHO Ten threats to global health in 2019: Antimicrobial resistance

Protocol: Estimating vaccine confidence levels among healthcare students and staff of a tertiary institution in South Africa: protocol of a crosssectional survey

Twitter Speaks: An Analysis of Australian Twitter Users' Topics and Sentiments About COVID-19 Vaccination Using Machine Learning

Measuring vaccine hesitancy: Field testing the WHO SAGE Working Group on Vaccine Hesitancy survey tool in Guatemala

Diagnosing the determinants of vaccine hesitancy in specific subgroups: The Guide to Tailoring Immunization Programmes (TIP)

Current tools available for investigating vaccine hesitancy: a scoping review protocol

Pre-pandemic cognitive function and COVID-19 vaccine hesitancy: cohort study

No psychological vaccination: Vaccine hesitancy is associated with negative psychiatric outcomes among Israelis who received COVID-19 vaccination

Social media and vaccine hesitancy

Assessing vaccine hesitancy in the UK population using a generalized vaccine hesitancy survey instrument

I Took the Trouble to Make Inquiries, So I Refuse to Accept Your Instructions": Religious Authority and Vaccine Hesitancy Among Ultra-Orthodox Jewish Mothers in Israel

Go-Win: COVID-19 Vaccine Supply Chain Smart Management System using BlockChain, IoT and Cloud Technologies

An empirical assessment of the intrusiveness and reasonableness of emerging work surveillance technologies in the public sector

Digital technologies in the public-health response to COVID-19

The need for privacy with public digital contact tracing during the COVID-19 pandemic

Computer audition for healthcare: Opportunities and challenges

Digital technology use during COVID-19 pandemic: A rapid review

Iot Network Identity Management Using Smart Contract and Blockchain Technology

An experimental study of fog and cloud computing in CEP-based Real-Time IoT applications

Artificial Intelligence of COVID-19 imaging: a hammer in search of a nail

Coronavirus disease (COVID-19) cases analysis using machine-learning applications

Predicting Vaccine Hesitancy and Vaccine Sentiment Using Topic Modeling and Evolutionary Optimization

COVID-19 vaccine hesitancy and related fears and anxiety

Examining Vaccine Hesitancy in Sub-Saharan Africa: A Survey of the Knowledge and Attitudes among Adults to Receive COVID-19 Vaccines in Ghana, Vaccines

The four dimensions of social network analysis: An overview of research methods, applications, and software tools

Artificial intelligence-enabled analysis of public attitudes on facebook and twitter toward covid-19 vaccines in the united kingdom and the united states: Observational study

Sentiment analysis based on deep learning: A comparative study, Electronics

Vaccine Hesitancy on Social Media: Sentiment Analysis from

Sentiment analysis algorithms and applications: A survey

A Review of Data Analysis for Early-Childhood Period: Taxonomy, Motivations, Challenges, Recommendation, and Methodological Aspects

Sentiment analysis and its applications in fighting COVID-19 and infectious diseases: A systematic review, Expert systems with applications

Using systematic review methods within a Ph. D. dissertation in political science: challenges and lessons learned from practice

Ten Years of Visualization of Business Process Models: A Systematic Literature Review

A systematic review of PIN-entry methods resistant to shoulder-surfing attacks, computers & security

Telehealth Utilization during the Covid-19 Pandemic: A Systematic Review

Identifying Polarity in Tweets from an Imbalanced Dataset about Diseases and Vaccines Using a Meta-Model Based on

Facebook and Twitter vaccine sentiment in response to measles outbreaks

Sentiment Analysis Methods for HPV Vaccines Related Tweets Based on Transfer Learning

Using Facebook to increase coverage of HPV vaccination among Danish girls: An assessment of a Danish social media campaign

Leveraging machine learning-based approaches to assess human papillomavirus vaccination sentiment trends with Twitter data

A natural language processing framework to analyse the opinions on HPV vaccination reflected in twitter over 10 years

A systematic literature review to examine the potential for social media to impact HPV vaccine uptake and awareness, knowledge, and attitudes about HPV and HPV vaccination

Tweeting on COVID-19 pandemic in South Africa: LDA-based topic modelling approach

Systematic Delineation of Media Polarity on COVID-19 Vaccines in Africa: Computational Linguistic Modeling Study

Tracking COVID-19 discourse on twitter in North America: Infodemiology study using topic modeling and aspect-based sentiment analysis

Analyzing the attitude of Indian citizens towards COVID-19 vaccine-A text analytics study

Thought I'd Share First" and Other Conspiracy Theory Tweets from the COVID-19 Infodemic: Exploratory Study

Applying Machine Learning to Identify Anti-Vaccination Tweets during the COVID-19

Evidence of disorientation towards immunization on online social media after contrasting political communication on vaccines. Results from an analysis of Twitter data in Italy

Social BI to understand the debate on vaccines on the Web and social media: Unraveling the anti-, free, and pro-vax communities in Italy

Adapting and extending a typology to identify vaccine misinformation on Twitter

Monitoring stance towards vaccination in twitter messages

Methods for social media monitoring related to vaccination: systematic scoping review

Vaccine hesitancy and illness perceptions: comparing parents of children with autism spectrum disorder to other parent groups, Children's Health Care

Examining emergent communities and social bots within the polarized online vaccination debate in Twitter

Semantic network analysis of vaccine sentiment in online social media

Sentiment analysis of social media on childhood vaccination: development of an ontology

Vaccines for pregnant women…?! Absurd"-Mapping maternal vaccination discourse and stance on social media over six months

Twitter as a sentinel tool to monitor public opinion on vaccination: an opinion mining analysis from

Using text mining and sentiment analysis to analyse YouTube Italian videos concerning vaccination

How do Twitter users react to TV broadcasts dedicated to vaccines in Italy?

Media news on vaccines and vaccination: The content profile, sentiment and trend of the online mass media during 2015-2020 in India

Sentiment Analysis of Social Media Data in Vaccination

Harnessing Social Media in the Modelling of Pandemics-Challenges and Opportunities

Using text mining to compare online pro-and anti-vaccine headlines: Word usage, sentiments, and online popularity, Communication studies

Exploring childhood anti-vaccine and pro-vaccine communities on twitter-a perspective from influential users

Prevalence of health misinformation on social media: systematic review

J o u r n a l P r e -p r o o f J o u r n a l P r e -p r o o f

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.