key: cord-0888588-87rm6m51
authors: Knight, Holly; Jia, Ru; Ayling, Kieran; Bradbury, Katherine; Baker, Katherine; Chalder, Trudie; Morling, Joanne R.; Durrant, Lindy; Avery, Tony; Ball, Jonathan; Barker, Caroline; Bennett, Robert; McKeever, Tricia; Vedhara, Kavita
title: Understanding and Addressing Vaccine Hesitancy in the Context of COVID-19: Development of a Digital intervention
date: 2021-10-18
journal: Public Health
DOI: 10.1016/j.puhe.2021.10.006
sha: d84f0560ab8ce476ae02252a427e3eaacfdead89
doc_id: 888588
cord_uid: 87rm6m51

Objectives Severe Acute Respiratory Coronavirus 2 (SARS-CoV-2) was identified in late 2019, spreading to over 200 countries and resulting in almost two million deaths worldwide. The emergence of safe and effective vaccines provides a route out of the pandemic, with vaccination uptake of 75-90% needed to achieve population protection. Vaccine hesitancy is problematic for vaccine rollout; global reports suggest only 73% of the population may agree to being vaccinated. As a result, there is an urgent need to develop equitable and accessible interventions to address vaccine hesitancy at the population level. Study Design & Method: We report the development of a scalable digital intervention seeking to address COVID-19 vaccine hesitancy and enhance uptake of COVID-19 vaccines in the UK. Guided by motivational interviewing (MI) principles, the intervention includes a series of therapeutic dialogues addressing 10 key concerns of vaccine hesitant individuals. Development of the intervention occurred linearly across four stages. During stage 1, we identified common reasons for COVID-19 vaccine hesitancy through analysis of existing survey data, a rapid systematic literature review, and public engagement workshops. Stage 2 comprised qualitative interviews with medical, immunological, and public health experts. Rapid content and thematic analysis of the data provided evidence-based responses to common vaccine concerns. Stage 3 involved the development of therapeutic dialogues through workshops with psychological and digital behaviour change experts. Dialogues were developed to address concerns using MI principles, including embracing resistance and supporting self-efficacy. Finally, stage 4 involved digitisation of the dialogues and pilot testing with members of the public. Discussion The digital intervention provides an evidence-based approach to addressing vaccine hesitancy through MI principles. The dialogues are user-selected, allowing exploration of relevant issues associated with hesitancy in a non-judgmental context. The text-based content and digital format allow for rapid modification to changing information and scalability for wider dissemination.

Severe Acute Respiratory Coronavirus 2 (SARS-CoV-2) was identified in late 2019. At the time of writing, the latest estimates suggest that it has spread to over 200 countries and has resulted in the deaths of almost two million people. 1 The resulting global pandemic has seriously affected the social and economic fabric of societies everywhere and the physical and mental health crisis continues. 2 Safe and effective vaccines provide a route out of this crisis, but the development of these vaccines, while necessary, are not sufficient. For vaccines to achieve their full potential, the public also need to be willing to be vaccinated. Recent data suggest this cannot be assumed. A recent survey of United Kingdom (UK) households indicated that approximately 82% of the population would agree to being vaccinated. 3 This estimate should be viewed against a backdrop of declines in vaccine intent overall and the fact that it masks large variations in intent between demographic groups. Vaccine hesitancy, defined as a "delay in acceptance or refusal of vaccines despite availability of vaccine services" 4 may significantly impact uptake of COVID-19 vaccines, particularly amongst ethnic minorities, women, and those with less education. 3, 5, 6 If, as has been suggested, 75-90% of a population will need to be vaccinated for community protection to be achieved, 7 then there is an urgent need to develop equitable and accessible interventions to address vaccine hesitancy at the population level within the UK.

Attempts to improve vaccine uptake are not new and have focussed traditionally on approaches such as information/education, incentives [8] [9] [10] and reminders. However, results from successive reviews suggest that the evidence-base in support of any one approach remains limited. [8] [9] [10] [11] Furthermore, much of the work has been conducted in the context of J o u r n a l P r e -p r o o f adults making decisions for their dependents, rather than adults making decisions for themselves. The generalisability of these findings to COVID-19 vaccines in adults is, therefore, unclear. Nonetheless, much can be gleaned from the existing evidence:

information, while necessary, is unlikely to improve vaccine uptake on its own, and interventions need to engage with individuals' reasons for hesitancy i.e., their hesitancy cognitions. 12 We report here a roadmap for the development of a scalable digital intervention which seeks to address the concerns of individuals who are vaccine hesitant, with a view to enhancing the uptake of COVID-19 vaccines. We report the process we followed in developing a digital vaccine hesitancy intervention suitable for adults considering a COVID-19 vaccination.

Whilst primary data were collected within the UK, we also drew from global evidence on vaccine hesitancy. In view of the urgency of the public health need, our approach to intervention development was pragmatic and took advantage of existing data where possible and appropriate. Evaluation of the intervention is underway and will be the focus of future work.

Our development involved four main stages and included involvement of public and patient partners throughout:

Stage 1: In order to understand and identify common reasons for COVID-19 vaccine hesitancy and acceptance we carried out a) an analysis of existing survey data collected within the UK during the pandemic, b) a rapid systematic literature review drawing from international literature and c) an examination of qualitative findings from a series of public engagement workshops regarding views of the public to immune challenges and vaccines.

J o u r n a l P r e -p r o o f Stage 2: We synthesised evidence from independent experts. This entailed qualitative interviews with experts from a range of relevant disciplines to identify evidence-based responses to the most common vaccine concerns raised by the public identified in stage 1. Stage 3: We developed 'therapeutic dialogues' to address common vaccine hesitancy concerns. These were developed in a workshop bringing together experts in psychological and digital behaviour change interventions.

Stage 4: The digital intervention was developed.

As this was a linear process with each stage informing the next, we present the methods and results from each stage consecutively.

Stage 1: Understanding and identifying common reasons for hesitancy & acceptance.

As part of a separate study into the UK population's mental and physical health over the course of the pandemic, we collected data regarding COVID-19 vaccination intention between 11 th -30 th November 2020 during the second national lockdown and prior to rollout of the vaccines (3 rd December 2020). During this period, daily case rates peaked at 24,962 (15 th November 2020) 14 . A detailed description of this study and the recruitment processes has been published previously 13 . But in relation to vaccine intention, respondents were asked "If you were offered a COVID-19 vaccine, would you take it?" and also asked, through a free text response, to elaborate on their main reason(s) for this intention. This item gave participants space to provide single or multiple responses, all of which were coded and analysed for common themes. One researcher (RJ) conducted a preliminary review of the free text data, allowing generation of initial themes. To enhance reliability, a second researcher (KA) independently examined the emerging themes, allowing further refinement. The frequency at which these themes appeared was quantified. Where vaccine hesitancy was J o u r n a l P r e -p r o o f indicated, themes were categorised within the WHO 3Cs model of vaccine hesitancy, which proposes that three main factors influence the decision to accept vaccines: confidence, complacency, and convenience 4 . All coding and categorisation was conducted with high levels of initial agreement (91% for reasons associated with vaccine hesitancy and 85% for reasons associated with agreement to vaccination). All discrepancies were resolved by discussion.

A total of n=762 individuals provided data (22% of whom indicated they were hesitant about receiving a COVID-19 vaccination); 93% (n=709) of respondents also provided a free-text response indicating their reasons for vaccine acceptance or hesitancy, of which 96% (n=683) provided sufficient detail for reasons to be categorised into themes. For those who expressed vaccine hesitancy, the most common concerns were found to map on to the WHO 3C category of 'confidence' (e.g., concerns related to long term complications, side effects and insufficient testing of the vaccines). The second most common concerns related to 'complacency' (e.g., beliefs of low personal risk of COVID-19, beliefs in ability to fight off the infection naturally). Concerns related to the 'convenience' category were the least common, but where they occurred they centred on a lack of information about the vaccines and altruism (i.e., other people needing the vaccines more) (see Table 1a ). In contrast, in respondents who indicated they would be willing to receive a COVID-19 vaccine, common reasons given related to 'self-protection', followed by 'hope to end the pandemic/wish for normal life' and a desire to 'protect the population or unspecified others and control the virus' (see Table 1b).   <INSERT TABLE 1A & TABLE 1B> J o u r n a l P r e -p r o o f

To identify additional themes/reasons for COVID-19 vaccine hesitancy that may have not been captured in our survey, a rapid systematic literature review was conducted. Four electronic databases (Medline, PsychInfo, Medrxiv, PsyAxiv) were searched to identify peerreviewed journal articles and pre-prints which examined reasons for COVID-19 vaccine hesitancy dated between 01/01/2020 and 03/12/2020: using the following search terms:

(COVID-19 vaccine hesitancy) OR ((COVID-19) AND (Vaccine hesitancy)). Randomised controlled trials (RCTs), mixed methods trials, cohort, and qualitative studies with adult participants were included. One researcher (RJ) conducted abstract and full-text screening to determine eligibility and a second cross-checked all eligibility decisions (KA).

Following title and abstract screening, 49 articles remained for full-text screening, with 10 ultimately deemed suitable for inclusion summarised in Table 2 . [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] The primary reason for excluding articles at the full-text screening stage were that many studies looked at vaccine intention only, not reasons for vaccine hesitancy (see Figure 1) . Three of the studies were (see table 2 ).

<INSERT TABLE 2>

Of the initial survey sample, 9.5% were from ethnic minority groups. To supplement the views of ethnic minorities captured in the survey, we also consulted PPI findings available through the University Hospital Southampton NHS Foundation Trust PPI team regarding the acceptability of vaccines. Several PPI meetings were held on this broad area between July-October 2020, including meetings that specifically sought the views of Black, Asian and Minority Ethnic (BAME) individuals.

The feedback from all the consultation meetings was reviewed and was found to reveal considerable overlap in the vaccine concerns identified in these meetings, with those identified as part of our survey and literature review. The only additional concerns related to whether vaccines had been tested on people from different ethnic groups and issues of trust in the medical and scientific communities. These issues were, therefore, prioritised for inclusion in our intervention.

The evidence emerging from the survey, rapid literature review and PPI findings was then triangulated through discussion between the two behavioural scientists (RJ, KA) contributing to this stage of the work. The aim of these discussions was to identify the most common J o u r n a l P r e -p r o o f COVID-19 vaccine concerns. This was based in part on the frequency with which concerns were identified in the survey, review, and PPI findings; ensuring that all three domains of the WHO 3C model were represented and that any unique perspectives raised by ethnic minority participants were also captured.

This led to the identification of nine core COVID-19 vaccine concerns. Concerns that were endorsed by fewer than 0.5% of the sample and did not align with concerns identified within the literature and PPI groups were not included within the intervention (i.e. vaccination is 'inconvenient'; table 1a.). In keeping with the most frequently cited concerns being related to 'confidence', 5/9 concerns related to 'confidence' (i.e., generalisability of evidence on vaccine safety and effectiveness to diverse populations; side-effects; rapid nature of vaccine development; clinical effectiveness and vaccine scepticism). Two out of nine concerns related to 'complacency' (i.e., low perceived risk of COVID-19 and belief in ability to fight off the infection naturally). A further two concerns related to 'convenience' (i.e., perceived lack of knowledge about COVID-19 vaccine and altruistic beliefs regarding others having a greater need). A tenth concern was subsequently added when the UK government decided to alter the dosing schedule from 3/4 weeks to up to 12 weeks between the two doses recommended for the Astra Zeneca and Pfizer vaccines. In keeping with the WHO 3C model, this latter issue also related to the issue of 'confidence'. Each theme/concern was given equal weighting within the subsequent development process.

Following the identification of 10 core vaccine concerns (Table 3) we sought to gather evidence-based responses to these concerns. This was achieved through semi-structured interviews with six academic and clinical experts from the fields of public health, general medicine, respiratory medicine and immunology with particular expertise in COVID-19 and/or COVID-19 vaccines. Each expert was presented with the list of 10 concerns and asked J o u r n a l P r e -p r o o f to provide an evidence-based response to each concern based on their knowledge of the scientific literature at that time. Interviews with experts were subjected to rapid thematic and content analysis after each interview and interviews continued until saturation in responses was achieved (i.e., no new responses emerged). 25 The expert responses demonstrated significant thematic overlap and consistency. Table 3 summarises the areas of evidence cited by experts in response to each concern.

<INSERT TABLE 3>

Our approach to developing the intervention was predicated on two main observations of the existing evidence. First that psychoeducation alone (i.e. provision of information gathered in MI is known to be effective in such contexts. 27, 28 Thus, for each of the most common vaccine concerns identified in Stage 1 we developed a therapeutic dialogue which would both impart information relevant to the individual concern, but do so using the communication principles of MI with a view to facilitating cognitive and, in turn, behaviour change i.e., reduce hesitancy and improve vaccine uptake. An online format was chosen to deliver the therapeutic dialogue to maximise audience reach and engagement, supported by a substantial evidence base on the use of this modality to promote vaccine uptake. 29, 30 Development of the therapeutic dialogues occurred through several expert workshops with behavioural scientists with expertise in MI, therapeutic interventions, digital interventions, behaviour change and COVID-19. First, key themes identified in the expert interviews (Stage 2) were discussed and translated into conversational language. The investigators chose a conversational approach to align with the online delivery format and to ensure inclusivity for all reading/English levels (see stage 4 below). Second, the dialogues were reviewed to identify points at which MI techniques could be integrated throughout. This process drew on contributors' experience in behaviour change research and adopted the approach proposed by Rollnick and colleagues. 26 This included expressing empathy through use of accepting and non-judgemental language. Developing discrepancy by simultaneously providing information related to the concern and presenting a rationale for vaccine uptake. The latter were derived from survey respondents willing to accept a COVID-19 vaccine (see Table 1b ) and sought to develop a discrepancy between the individual's cause for concern and their wider personal values and goals. Embracing resistance by acknowledging that their concerns are shared by others and are legitimate and supporting self-efficacy by reinforcing the individual's personal J o u r n a l P r e -p r o o f agency in making their decision to accept a vaccine or not. See Table 4 for illustrative examples of how MI principles were embedded within the therapeutic dialogues.

Finally, we hosted a PPI workshop to discuss the resulting dialogues. Participants were members of the general public recruited through the University Hospital Southampton NHS 

The script from each of the 10 therapeutic dialogues provided the architecture for our digital, web-based, vaccine hesitancy intervention. Given high rates of internet usage throughout the UK (92% of adults) 31 and other similar developed countries, it was felt that use of a digital platform would maximise reach and accessibility. The research team worked with a digital development company to design and build a conversational interface through which individuals identify the issue that most closely underpins their reason for being hesitant, J o u r n a l P r e -p r o o f (from the issues stated above e.g., concerns about side effects). This identification triggers an MI driven therapeutic dialogue relevant to the selected concern, with opportunities for the individual to further explore the content as they progress through the dialogue as well as to access responses to more than just their initial concern.

Once developed, the digital intervention was piloted with 18 members of the public (nine male / nine female) who had no previous experience with the dialogues. Participant feedback on the dialogue content, user interface, accessibility, and general presentation led to a final iteration of the intervention, which can be viewed here: www.covidvaxfacts.info. For illustrative screen shots, see supplementary figures 1 and 2.

The development of safe and effective vaccines against SARS CoV-2, while necessary, will not be sufficient to contain COVID-19 unless we also achieve high vaccine uptake. We have described here the rapid development of an evidence-based digital intervention which draws on the communication principles of MI and is in keeping with many of the recommendations made in a recent review of approaches to increasing vaccine uptake e.g., focus on the concerns of the population. 32 Our aim is to provide the end-user with an intervention which is individualised to their specific concerns, acknowledges the legitimacy of these concerns, provides up to date information related to these concerns whilst also providing an accepting non-judgemental context in which they can explore their reasons for hesitancy. The textbased content and digital format mean it can be readily scaled-up for wider dissemination and rapidly modified for implementation in different languages and to respond to changing information.

Although this intervention, like much else to do with COVID-19, has been developed at pace we think the process highlights some potential issues regarding intervention development worthy of discussion. First, the development of our digital, behavioural intervention followed a fairly conventional path as outlined in the Medical Research Council's (MRC) best practice guidance. This involved evaluating the evidence base and theory as well as incorporating the views of target users (i.e. members of the general public). 33 This was possible partly because we had timely access to PPI findings available through the University Hospital Southampton NHS Foundation Trust regarding the acceptability of vaccines, allowing rapid comparison of the PPI findings with the concerns identified through our existing survey data and literature review.

A critical step in digital intervention development is optimisation of intervention content, since digital intervention content cannot be adjusted 'in the moment', like in a practitioner delivered intervention. We were able to conduct optimisation work with PPI, albeit with a smaller sample (N=4) than might usually be employed in digital intervention development.

Computer science methodology states that during intervention optimisation around 80% of views can be captured with five target users and we were close to this threshold. 34 However, best practice guidance from digital health psychology suggests including larger, diverse samples is important to ensure views of people from different backgrounds are considered. 33 Despite having a smaller sample, our optimisation with PPI did help us to improve the persuasiveness and accessibility of the key messages within the intervention. It is possible that we may have found other important ways of optimising our content by including a larger, more diverse group of PPI at this stage. However, it is important to note that this intervention is quite simple, it targets only one behaviour, draws on a very well-established behavioural technique which guided content design (MI), and it addressed barriers that were thoroughly identified using existing evidence in the intervention planning stage. Therefore, in this J o u r n a l P r e -p r o o f particular context, it is possible that sufficient optimisation was achieved with a smaller sample. Following launching the intervention, we were able to remain responsive to changes in the vaccine guidance by seeking expert advice in relation to risk of blood clots, protection against emergent variants, pregnancy, and vaccination during Ramadan. Expert responses on these topics were compiled into a breaking news section and updated within the dialogues in line with government recommendations. Delivering the intervention digitally provides a rapid means of evaluating and evolving the intervention as reasons for hesitancy change, allowing real-time collection of both analytics and usage data alongside data that might answer specific research questions.

The MRC highlights the importance of making use of existing data and evidence wherever possible. In this work, we were able to benefit from data collected as part of another study 13 where we were able to identify specific concerns related to vaccine hesitancy. We also drew on evidence kindly shared with us by others. This allowed acceleration of the intervention development and improved the economic efficiency of research.

In view of the urgency of the public health issue we conducted a rapid review. Given the rapid evolution of the scientific landscape, we acknowledge that new work may have since emerged. However, to the authors knowledge, recent work provides evidence on the persistence of the primary concerns underpinning this intervention within the general public. 23, 35, 36 Indeed, COVID-19 has most likely led to an unprecedented number of rapid reviews, as the scientific community have clamoured to understand the available evidence as quickly as possible. Although, it is clear that rapid reviews take many forms (e.g., limited by language, dates, databases etc.), they do vary in the quality of their reporting and the methodological shortcuts they take. 37 The implications of these inconsistencies for the quality and validity of these reviews is, however, unclear as there is thus far limited evidence comparing the results of different review approaches. The provision of such evidence in J o u r n a l P r e -p r o o f future research would undoubtedly inform the contexts in which it is appropriate to conduct rapid reviews, and the methods that should be employed. Such guidance now exists for scoping reviews. 38 and would appear to be in development for rapid reviews by the Equator network. 39

Whilst we have attempted to create an intervention that is scalable, limitations to our work are noted. Although development of the intervention was predicated on findings from the international literature and incorporated feedback from an ethnically diverse PPI group, the survey data used in Stage 1 was collected from a predominantly white sample (90.3%) within the East Midlands 13 . Given elevated rates of vaccine hesitancy amongst ethnic minority groups, a targeted approach to the development and rollout of future interventions is warranted. Additionally, participants were asked to report their own concerns about receiving a vaccine however, these concerns may have changed over the course of vaccine rollout and be influenced by an individual's social networks and the media. Exploring temporal changes to vaccine hesitancy and the impact of external factors on intrinsic concerns is a worthwhile avenue for future research. Finally, whilst the utility of delivering an intervention digitally is relevant for countries with high internet usage, the mechanism for delivery in less developed countries requires careful consideration. However, it is hoped that this paper provides a framework for future iterations of rapid behavioural interventions, which can be adapted to meet the unique needs of the population and behaviour of choice.

In summary, for COVID-19 vaccines to achieve their full public health potential, the public need to be willing to be vaccinated. Recent data suggest this cannot be assumed. We have reported here on the development of a scalable digital intervention which seeks to address the J o u r n a l P r e -p r o o f concerns of individuals who are vaccine hesitant with a view to enhancing their confidence in COVID-19 vaccines and, in turn their uptake. The effects of the intervention on these outcomes will be the subject of future work. Complacency regarding whether the disease is common; that the immune system is sufficient to fight off the disease and the disease is not severe.

Constraints to getting vaccinated such as everyday stress;

inconvenience; visiting the doctors; discomfort.

Calculations involving weighing up benefits and risks; needing to closely consider whether it is personally useful; needing to understand more about vaccines and vaccination. Concerns about vaccine safety (100%) centred on the newness of the vaccine and its safety (e.g., long-term effect, side effects) and effectiveness.

a Themes or responses were based on participants who provided information on vaccine hesitancy.  Delaying the second dose from 3 to 12 weeks also gives the immune system longer to develop immunity.

 In the Oxford-AstraZeneca vaccine trials, a longer gap between doses offered better protection. J o u r n a l P r e -p r o o f 

The mental health impact of the COVID-19 pandemic on people with and without depressive, anxiety, or obsessive-compulsive disorders: a longitudinal study of three Dutch case-control cohorts

Predictors of COVID-19 vaccine hesitancy in the UK Household Longitudinal Study

The Sage Working Group

Anti-vaccine attitudes and risk factors for not agreeing to vaccination against COVID-19 amongst 32,361 UK adults: Implications for public health communications

Measuring the Impact of Exposure to COVID-19 Vaccine Misinformation on Vaccine Intent in the UK and US

Challenges in creating herd immunity to SARS-CoV-2 infection by mass vaccination

Improving vaccination uptake among adolescents

Effectiveness of email-based reminders to increase vaccine uptake: a systematic review

Strategies for increasing uptake of vaccination in pregnancy in high-income countries: A systematic review

Strategies intended to address vaccine hesitancy: Review of published reviews

Addressing common barriers in adult immunizations: a review of interventions

Mental health in the UK during the COVID-19 pandemic: cross-sectional analyses from a community cohort study

Cases in United Kingdom

When it is available, will we take it? Public perception of hypothetical COVID-19 vaccine in Nigeria

Attitudes Toward a Potential SARS-CoV-2 Vaccine : A Survey of U.S. Adults

Acceptance and preference for COVID-19 vaccination in health-care workers (HCWs)

Assessment of COVID-19 vaccine acceptance among healthcare workers in Los Angeles

Vaccine hesitancy in Maltese family physicians and their trainees vis-à-vis influenza and novel COVID-19 vaccination

Sharp rise in vaccine hesitancy in a large and representative sample of the French population: reasons for vaccine hesitancy

Are we ready when COVID-19 vaccine is available? Study on nurses' vaccine hesitancy in Hong Kong

Influences on Attitudes Regarding Potential COVID-19 Vaccination in the United States. Vaccines (Basel)

vaccination intention in the UK: results from the COVID-19 vaccination acceptability study (CoVAccS), a nationally representative cross-sectional survey

Towards intervention development to increase the uptake of COVID-19 vaccination among those at high risk: Outlining evidence-based and theoretically informed future intervention content

Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration

Motivational interviewing in health care. Helping patients change behavior

Motivational interviewing for improving outcomes in youth living with HIV. Cochrane Database Syst Rev

Motivational interviewing: A powerful tool to address vaccine hesitancy

Dose-Response Relationship of a Web-Based Tailored Intervention Promoting Human Papillomavirus Vaccination: Process Evaluation of a Randomized Controlled Trial

Efficacy of a Web-Based Intervention to Increase Uptake of Maternal Vaccines: An RCT

A Rapid Systematic Review of Public Responses to Health Messages Encouraging Vaccination against Infectious Diseases in a Pandemic or Epidemic. Vaccines (Basel)

The person-based approach to intervention development: application to digital health-related behavior change interventions

Estimating the number of subjects needed for a thinking aloud test

the UK: the Oxford coronavirus explanations, attitudes, and narratives survey (Oceans) II

Confidence and Receptivity for COVID-19 Vaccines: A Rapid Systematic Review. Vaccines (Basel)

A scoping review of rapid review methods

PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation

We would like to thank all experts for their contributions to the expert interviews. 

Records identified through database searching (n = 49)

Additional records identified through other sources (n = 0)Records after duplicates removed (n = 48)Records screened (n = 48)Records excluded (n = 10)Full-text articles assessed for eligibility (n = 38)Full-text articles excluded, with reasons (n = 28)  did not investigate reasons for COVID-19 vaccine hesitancy (n=27)  experiment (n=1) Studies included in qualitative synthesis (n = 10)