key: cord-0760800-w82p4lx4
authors: Arghittu, Antonella; Deiana, Giovanna; Dettori, Marco; Dempsey, Emma; Masia, Maria Dolores; Palmieri, Alessandra; Spano, Antonio Lorenzo; Azara, Antonio; Castiglia, Paolo
title: Web-based analysis on the role of Digital Media in Health Communication: the experience of VaccinarSinSardegna Website
date: 2021-10-01
journal: Acta Biomed
DOI: 10.23750/abm.v92is6.12072
sha: 817eeea55bf75b05cd28184b7d24e5f8709770f6
doc_id: 760800
cord_uid: w82p4lx4

BACKGROUND AND AIM: In 2013, in a bid to tackle vaccine hesitancy, the Italian Society of Hygiene and Preventive Medicine (SItI) created the national VaccinarSì project, a vaccination communication website. Following the success of the national portal, several Italian Regions began to develop regional versions of the website (VaccinarSì Network). One of the regional websites (VaccinarSinSardegna.org) was set up in the Region of Sardinia, in doth Italian and English, with the aim of communicating and educating the general public on the importance of vaccines and their health and social value. This article aims to present the structure of the Vaccinarsinsardegna.org website, to demonstrate the actions carried out to provide accurate information for users and to analyse the website’s activity from its launch until the current pandemic period in terms of dissemination and visibility. METHODS: Various metrics such as the number of visits to the site (sessions, number of users and average session duration), user behaviour (pages viewed, bounce rate and organic search) and the session acquisition path (direct traffic, referrals and social traffic) were recorded, extrapolated and processed with Google Analytics. Qualitative and normally distributed quantitative variables were summarised with absolute (relative) frequencies and means. The statistical differences between the two periods (before and during the anti-COVID-19 vaccination campaign) were evaluated applying the Wilkoxon runk-sum test. A two-tailed p-value less than 0.01 was considered to be statistically significant. RESULTS: The metrics recorded using Google Analytics show that from its publication to date, the VaccinarSinSardegna.org portal has significantly increased its visibility: in fact, the number of users who have accessed the site, the number of sessions and individual web pages visited have grown exponentially. These findings are confirmed by the increasing number of visits to both versions of the regional website (Italian- and English-language) during the COVID-19 emergency and COVID-19 vaccination campaign. When the total values recorded over the period were compared, an overall increase in metrics was observed—the number of individual users, visits and individual pageviews rose in a statistically significant way. CONCLUSIONS: Based on the success on VaccinarSinSardegna and given its potential for expansion to include further collaborations with national and international scientific bodies, the authors affirm that similar initiatives should be implemented in the fight against vaccine hesistancy. (www.actabiomedica.it)

Vaccination is one of the most important scientific breakthroughs in the history of medicine and has made a major contribution to increasing life expectancy (1, 2) . Vaccines have led to the eradication of smallpox, a dramatic reduction in the incidence of debilitating diseases such as polio and measles, and they have been shown to be effective in the prevent of certain cancers (3) (4) (5) (6) (7) . The economic, medical and social burden of preventable infectious diseases has greatly diminished thanks to the success of vaccines and vaccinations. However, despite the scientific consensus on the safety and efficacy of vaccinations, there is still a discrepancy between the scientific evidence on their validity and the public's perception of the risk attributed to them (8) (9) (10) (11) (12) . Since people no longer directly experience the diseases which have already been eradicated by vaccines, parents especially do not feel the need to immunise themselves and their children (13, 14) .

Parents and patients, more concerned about vaccination risks than about the natural disease and its complications, sometimes choose to delay vaccination or request alternative vaccination programmes. This phenomenon, known as vaccine hesitancy, has been further fuelled by misinformation spread on the Internet (15) (16) (17) . The decision-making process regarding health choices, in fact, is strongly influenced by what one reads online, especially in relation to issues such as vaccination (18) (19) (20) . In particular, the presence of anti-vaccination content on the web has influenced the perception of the risk linked to vaccines, with a consequent drop in vaccination coverage among all segments of the Italian population (9, 12, 18) . Furthermore, anti-vaccination groups have improved the communication techniques they use to spread their theories by exploiting new digital information and communication technologies (ICTs) i.e.: blogs, websites, media, social media platforms (21, 22) .

In this regard, several studies show that, in decision-making processes such as adherence to vaccination programmes, the public seeks out collective knowledge that is readily available online. Given the potential of ICTs in information searching and sharing processes, for some time now health institutions have also been using online channels as tools for disseminating medical and scientific knowledge among users/patients (23) (24) (25) (26) . The elevated power of information sharing, in fact, has prompted Public Health to design and implement a series of online interventions with the aim of i) increasing the spread of high-quality reliable health information, ii) involving citizens/ patients, thereby giving them as much control as possible over their own health; iii) increasing awareness about infectious diseases and health treatments to prevent them (26) (27) (28) (29) .

In Italy in 2013, in a bid to tackle vaccine hesitancy, the Italian Society of Hygiene and Preventive Medicine (SItI) created the national VaccinarSì project, a vaccination communication website (29) (30) (31) . Following the success of the national portal, several Italian Regions began to develop regional versions of the website (VaccinarSì Network) (33) .

In the Italian health landscape, in fact, vaccination policies have been characterised by a strong territorial heterogeneity. As such, in accordance with Title V of Constitutional Law no. 3 of 18 October 2001 which allows Italian regions a certain degree of autonomy in the management of health systems (in particular in vaccination prevention programmes), information/ communication strategies on vaccination must also be devised according to the local context (34). With this in mind, the regional website VaccinarSinSardegna. org was set up in the Region of Sardinia with the aim of communicating and educating the general public on the importance of vaccines and their health and social value (33) . One of the website's specific objectives is to communicate with the public and disseminate evidence-based information on vaccines, offsetting the misleading and erroneous information which circulates online. In particular, one of the website's main activities is to provide, through a virtual counselling desk, information on vaccines and service delivery at regional and local level (35) .

On the basis of these premises, this article aims to present the structure of the Vaccinarsinsardegna.org website, to demonstrate the actions carried out to provide accurate information for users and to analyse the website's activity from its launch until the current pandemic period in terms of dissemination and visibility. The study also aims to describe user behaviour through an analysis of the number of visits and individual pageviews, the user satisfaction index (usefulness), the number of national and international requests for advice and/or information and the average consultation time. It also evaluates the strategies necessary to counteract cyber disinformation through the potential offered by ICTs.

This project envisaged the creation of a VaccinarSì website for each of the participating Italian Regions (35) . Subsequently, the project was included in the Region of Sardinia's 2014-2019 Regional Prevention Plan as a sentinel indicator of the programme for the development and reinforcement of vaccinations. The website is sponsored by the Italian Ministry of Health, Italian National Institute of Health (ISS), Italian Paediatric Federation (FIMP), Italian Federation of General Practitioners (FIMMG), Autonomous Region of Sardinia (RAS), Sardinian Health Protection Agency and Sassari University Hospital (AOU-SS). By interacting with users, the website plays a fundamental role in promoting health, assisting the public, bridging the gap between experts and citizens, fostering social relations, and internationalising services.

Its organisational structure consists of four units: (i) a directive unit (restricted board) composed of 2 experts in Hygiene and Preventive Medicine; (ii) a scientific committee composed of 15 public health professionals whose expertise guarantees the quality of the information published; and (iii) an operational committee composed of 8 experts (hygienists, infectologists, paediatricians and health communication professionals) whose task is to update information and provide advice/correspondence to users and (iv) a working group composed of 4 experts including 2 hygienists, a project manager and an English mother tongue expert.

To create the website, a preliminary analysis of the epidemiological framework on the national and international scene was carried out, with the aim of discerning the potential determinants of vaccination hesitancy and developing in-depth information adapted to fit the local context.

The website's content is divided into the following five macro-areas: i) Science and knowledge: an area structurally identical to the national site in terms of layout and content; ii) Vaccinations in Sardinia: adapted to the local context; iii) Highlights: dedicated to events, initiatives and news; iv) Useful information for citizens: giving the addresses and outpatient schedules of vaccination services throughout the region; v) Contacts: a virtual advice desk through which users can request information. The network is hosted on a dedicated server. The sections of each macro-area are schematically summarised in Table 1 .

In order to respond to the various information needs of users, the collection and drafting of the content present on the VaccinarSì Network is organised into three information levels for each topic covered: i) title and abstract; ii) short assay; iii) long article.

In addition, a newsletter system was put in place to enable those users who subscribe to be kept up-todate on published content.

On 18 March 2019, in order to gauge public satisfaction in terms of the usefulness of the content published, the user satisfaction function was introduced for each new article and/or news item published, and on 14 November of the same year the English version of the website was launched.

The website's activities are described from the publication date of VaccinarSinSardegna.org in Italian (04 November 2017) and in English (14 November 2019) until 30 June 2021.

The data was researched, extrapolated and processed with Google Analytics, which, using adaptive formulas on the data acquired by the software, allows the analysis of metrics such as: the number of visits to the site (sessions, number of users, average session duration), user behaviour (pageviews, bounce rate, organic searches) and the session acquisition path (direct traffic, referral and social traffic). A description of the macro-categories and metrics considered in the analysis (glossary) is given in Table 2 . 

Vaccine preventable diseases (VPDs)

This section contains a brief description of the infectious diseases for which vaccination is available.

Available vaccines:

This section explains the methods, rules and indications relating to the administration of a vaccine are dealt with in this section. It also contains a direct reference to the corresponding disease.

In this section, the benefits of vaccination and its future prospects are presented and discussed, alongside insights into the real risks related to vaccination practices.

The articles in this section analyse and respond to the main theories of the anti-vaccination movements, who operate mainly on the Internet spreading experimental evidence and news that is not supported by data. This results in misleading consequences, inaccurate results, illusory myths and urban legends: or in a word, misinformation.

In this section, the main issues involved in international travel and planned vaccinations are discussed.

This section briefly outlines the Region of Sardinia's 2014-2019 Regional Prevention Plan for development and reinforcement of vaccination.

The articles in this section describe the prevention and health promotion activities carried out at regional level and included in the regional preventive vaccination policies.

This section describes the vaccination offer schedule provided for in the national reference documents and adapted to the Sardinian context (Art. V of the Constitution).

In this section you can view the Decrees, Circulars and Recommendations concerning the recent introduction of mandatory vaccinations.

This section contains the annual circulars and recommendations regarding the flu vaccination campaign.

Tackling anti-microbial resistance (AMR)

In this section it is possible to consult all the available attachments concerning the activities implemented to counter AMR at national and international level, with a specific focus on the initiatives promoted by our Region.

The documents contained in this section are related to infectious disease surveillance systems and their enhancement in response to infectious emergencies.

The names and resumes of the members of the committees and panels of experts are contained in this section.

The names and resumes of the members of the working group are contained in this section.

News A communication task force is responsible for periodically publishing short news reports on the following topics: epidemiological data on vaccine-preventable infectious diseases (outbreaks and epidemics), vaccination guidelines and programmes, health education and health communication.

This section contains the main events on vaccination that are organised at regional level (i.e.: vaccination communication campaigns; public health congresses and conferences; direct meetings with citizens on the occasion of dedicated Awareness Days. 

Visits to the site Session A single period of continual active viewing by a user.

The number of people that have visited the site at least once during a given timeperiod. One user may have multiple sessions, but will still be counted as a single user.

The Average Session Duration gives a top-level view of how long users spend on the website.

The total number of website pages viewed. For example, if one user visited the homepage and the contact page, it counts as two pageviews.

% Bounce rate The percentage of sessions with a single pageview (an indicator of NON-transitability within the website).

Organic Traffic originating from natural (or unpaid) results on search engines.

Direct traffic includes people who typed the website's URL into their browser or who had previously memorised it in their browser.

A referral occurs when a user clicks through to the website from another third-party website (not social media).

Number of visits to the site via a link or reference from social media platforms.

An ad-hoc electronic form was drawn up to collect the main environmental variables using Excel (Microsoft Office, Microsoft Corporation, Redmond, WA, USA). Qualitative and normally distributed quantitative variables were summarised with absolute (relative) frequencies and means. The statistical differences between the two periods (before and during the anti-COVID-19 vaccination campaign) were evaluated applying the Wilkoxon runk-sum test. A two-tailed p-value less than 0.01 was considered to be statistically significant. The statistical computations were performed using STATA 16 (StatCorp., Austin, TX, USA).

The Italian-language VaccinarSinSardegna.org website was launched on 3 November 2017 as part of a Vaccine Communication project curated by SItI.

During the study period (03 November 2017-30 June 2021) approximately 166,000 users accessed the portal and 18% of these visited the site more than once. In total, the website's pages received more than 420,000 views, with an average session duration of 1.36 minutes. Accesses to the Italian version of the site and user behaviour in each month of activity for the study period are described in Table 3 .

In socio-demographic terms, most users came from within Italy (95%): Rome (14.4%), Cagliari (15.4%), Sassari (9.6%) and Milan (8.2%). As for those who accessed the site from abroad, most came from the United States (1.1%), United Kingdom (0.5%), Germany (0.3%), France (0.3%), Switzerland (0.2%), Spain (0.2%), China (0.2%), Brazil (0.1%) and (2.2%) from 110 other countries. The devices most commonly used to access the portal were mobile phones (62.2%), while personal computers (PCs) and tablets accounted for 33.8% and 4% respectively. Most visitors (86.8%) accessed the site through Organic Search, 9.4% through Direct, 2.9% through Referral and 0.9% through social networks (mainly Facebook). As regards visits to specific pages, the pages which had most visits were those related to the Vaccination Schedule and those linked to the most common fake news about vaccines.

The trend of consultations in the period considered is shown in Figure 1 .

In particular, in the last six months (01 January 2021-30 June 2021), in correspondence with the COVID-19 vaccination campaign conducted in Sardinia, there was an increase in interactions with the portal, which recorded over 36,000 consultations by over 30,000 users for over 77,000 page views in April alone. Overall, for the same metrics, the monthly average for the six-month period was 21,863, 18,300 and 48,319, compared to 1,777, 1,478 and 3,437 in the previous period with statistically significant differences (p<0.01) (Figure 2 ).

The English-language version of the VaccinarSin-Sardegna.org website was launched on November 04, 2019. From the publication date until 30 June 2021, 7,774 hits were recorded on the site with an average consultation time of 1.11 minutes. The number of pages viewed was 16,331 with a bounce rate of 68.2%.

Metrics relating to the English-language version of the site and user behaviour in each month of activity from the date of publication to 30 June 2021 are described in Table 4 .

In socio-demographic terms, all contacts came from abroad (100%); in particular, more than 65% of Table 4 . Metrics for the English-language version VaccinarSinSardegna.org website in the period of study (from publication date until 30 June 2021). hits were recorded as coming from a European country (Germany, UK, France, Switzerland) and 35% from other continents (United States, China, Brazil, Asia). The devices most commonly used to access the site were mobile phones (73.1%) followed by tablets (16.7%) and personal computers (10.2%). The trend of consultations in the period considered is shown in Figure 3 .

Notably, the English-language portal in the period corresponding to the anti-COVID-19 vaccination campaign (01 January 2021-30 June 2021) also recorded an increase in interactions in the following metrics: sessions, users and pageviews. In fact, these went from an average of 151 sessions, 126 users and 377 pageviews recorded from the date of publication to the pre-COVID-19 vaccination campaign period (04 November 2019-31December 2020) to an average of 944,859 and 1,843 respectively, recorded during the vaccination campaign (01 January 2021-30 June 2021) with statistically significant differences (p<0.01) (Figure 4) .

From the publication date (04 November 2019) to 30 June 2021, the portal recorded a total of 838 requests for advice and/or information via email. In fact, most of these were received during the last six months analysed, during which the communication task force replied to over 700 emails from 670 different users. The most frequent request concerned the times and contact details of the vaccination clinics for anti-COVID-19 vaccination and for childhood and adolescent vaccination booster shots (mainly MPR and HPV), as well as information as to how users can obtain the vaccination certificate for themselves or their children.

As regards the user satisfaction indicator concerning the usefulness of the published content, 2,697 interactions were recorded on 391 pages evaluated. Of these, 1,621 returned a positive satisfaction indicator in terms of usefulness of the published content and 1,076 returned a negative satisfaction indicator. In particular, the highest number of positive feedbacks was recorded in relation to the pages on: (i) vaccination clinics in Sardinia (66%), (ii) updating of the computerised Vaccine Registry (75%), (iii) currently available vaccines (61%), (iv) pages related to counter-information (83%), (v) news and events (90%), (iv) flu vaccination prescriptions (88%), (vii) news related to the surveillance of invasive bacterial diseases in Sardinia (70%), (viii) progress of the anti-COVID-19 vaccination campaign (50%). Negative satisfaction indicators were recorded for: i) the vaccination schedule (88%), ii) regional reports (67%), iii) mandatory vaccination (69%).

In addition, the website participated as an institutional communication tool of the Region of Sardinia in several health education projects and initiatives that were promoted at the regional level on the occasion of health promotion days (i.e. International HPV Awareness Day, World Immunisation Week, World Poliomyelitis Day, World Antimicrobial Awareness Week and European Antibiotic Awareness Day, Save Lives Clean Hands campaign and the communication campaigns for influenza vaccination).

The digital revolution has brought with it a new era in communication, profoundly changing the way people interact. Moreover, the current pandemic, and the resulting social isolation imposed on citizens, has further contributed to the increased use of digital platforms (36) .

About half of the world's population regularly uses social media: 5.19 billion people own a mobile phone and 4.54 billion people are regularly connected to the Internet (37).

With the shift to Web 2.0 and the social Web, the Internet user has gone from being a passive receptor with few opportunities for interaction to a more active player creating online content of various types and in various formats (e.g. articles, comments, videos) (16, 17, 38, 39) . This has led to an increase in the dissemination of misinformation, particularly as regards the subject of vaccination.

Indeed, the quality of information on the web varies widely. The considerable number of inaccurate or misleading messages appears to kindle a negative attitude (i.e. vaccine hesitancy) to a greater extent than institutional information channels promoting vaccination generate a positive attitude. This is even more worrying if one considers that the vaccine-related content on the web is not always well regulated, and the dissemination of incorrect and misleading information cannot be curtailed or monitored (16) . The increased information-sharing power and the spread of vaccine hesitancy have therefore forced Public Health to revolutionise the way it communicates with users/ patients (40) . Thus, while many approaches now exist to encourage people to accept vaccination (e.g. storytelling and celebrity endorsements etc), of these new tools the use of institutional websites promoted by scientific societies appears to have encouraging results in terms of improving user adherence to immunisation programmes (12, 13, 41, 42) . In these years of activity, the VaccinarSinSardegna.org website has actively and proactively interacted with users, playing a crucial role in health promotion, personal assistance, the relationship between experts and citizens, fostering social relations, and the internationalisation of services. In addition, by adopting the "setting for health" approach, which promotes multidisciplinary health education interventions (empowerment, dialogue and shared participation), it has provided for 'bundled' activities and initiatives with vaccination as the focus and which were tailored to the various contexts and levels of prevention. Examples of such activities include those regarding vaccinations in the fight against antimicrobial resistance on the occasion of World AMR Week, primary prevention through vaccination and hand hygiene on 5 May for World Hand Hygiene Day, and the prevention of cervical cancer with HPV vaccination. These events also helped raise awareness about the portal among the various stakeholders and the general public.

In particular, since its launch, the site has been promoting vaccination culture and constantly expanding its catchment area, with hits to the site coming from over 100 countries. Contacts from abroad may be the result of the website's inclusion in the VaccinarSì Network, which is in turn one of the 53 web portals which make up the Vaccine Safety Net. Indeed, this has contributed to the social growth and cultural orientation of the regional sites, enabling them to reach an extremely wide audience not only locally but also in a broader national and international spectrum (33) .

By virtue of this and of the numerous requests for information coming from abroad regarding travel medicine and the possibility of tourists being vaccinated in Sardinia, in December 2019 VaccinarSin-Sardegna.org was also published in English as a true pathfinder for the very recently launched European VaccinarSì network.

The metrics recorded using Google Analytics show that from its publication to date, the Vacci-narSinSardegna.org portal has significantly increased its visibility: in fact, the number of users who have accessed the site, the number of sessions and individual pageviews have grown exponentially. This result, which is in line with international findings (43) , may be linked to the growing need to find health information on the Internet, especially regarding vaccination, and to the numerous health promotion and education projects that have been promoted at regional level and in which the site has always participated as an institutional communication tool of the Region of Sardinia.

These findings are confirmed by the increasing number of visits to both versions of the regional website (Italian-and English-language) during the COVID-19 emergency when, in a six-month period (01 January 2021-30 June 2021), the number of pages viewed increased 14 times for the Italian version and over 5 times for the English-language version, compared to the average recorded from the date of publication of the two websites to 31 December 2020. Further confirmation comes from the increase in the number of requests received by email (120 per month vs 20 per month) in the same period. This increase in requests for advice could also be due to the population's need for feedback and an answer to their information needs in such a period of disorientation and despondency as the current pandemic (44) . The radical changes imposed by the pandemic, the necessary anti-COVID-19 measures, the rise of smart working and distance learning, but also a general increase in the time spent at home (45) (46) (47) , along with the need to minimise travel and contact with other people have led the population to spend a much greater part of the day on the Internet than in the past. Indeed, according to Zhao et al., the isolation measures imposed during the pandemic led to an increased use of digital social platforms to search for information (21, 22) . This also emerges from our observations, as there was also an increase in the use of mobile devices compared to desktop devices for accessing the website.

In this regard, several authors claim that during the COVID-19 pandemic, 80% of active web users were looking for health information and clear answers regarding the current health landscape and the advancement of the vaccination campaign. This data is confirmed by the metrics recorded by Vaccinarsinsardegna.org; in fact, in the last six months analysed (01 January 2021-30 June 2021), in correspondence with the COVID-19 vaccination campaign conducted in Sardinia, there was an increase in requests for advice (over 700), an increase in interactions with the portal, which recorded over 36,000 consultations by over 30,000 users for over 77,000 pageviews in April 2021 alone.

The VaccinarSinSardegna.org website has met users' need to seek information and find answers to their queries by giving exhaustive and informative responses to emails or by directing users to offices and/or specialist clinics within 24 hours of the request being received.

With regard to user behaviour, the period under review saw higher values of organic traffic, to the detriment of referral and social acquisition paths. This phenomenon confirmed the website as a privileged direct research tool to acquire health information, mainly on vaccines in Sardinia and, together with the contacts registered from outside the region (Rome, Milan), also confirmed that the site is now known to be a valuable direct research tool for acquiring health information, mainly on vaccines.

The portal's strong point is the publication of evidence-based topics, with a focus on local and regional information. News is published with a consistent frequency of one to two articles per month. In addition, the user satisfaction feature shows that for most users who returned feedback, the content is displayed in a clear and referenced manner, suggesting that the information published is useful and easy to understand. On the other hand, the negative satisfaction indicators recorded provide useful insights to allow the published content to be adapted to the needs of the users.

The peaks in the number of single users, sessions and pageviews are normally recorded during the autumn and winter seasons for all the years considered in correspondence with the school year (September-June) and the flu vaccination campaign (48) . This data could be attributable to the users' need for information about vaccination for adults (flu vaccination in particular) as booster shots are administered in that period of the year, and to parents' need for information about vaccination certificates required to enrol their children in school (and/or how to catch up missed vaccinations) in compliance with the national legislation on mandatory vaccinations (49) . Further peaks can be seen when news and/or updates are published, presumably due to the newsletter system that sends a notification to subscribed users every time a news article or an event is published.

However, while the increase in consultations during the last flu vaccination campaign was similar to that of previous flu seasons, the peak and average consultation values in the last winter were more than ten times higher during the COVID-19 vaccination campaign.

A key role in the website's success is played by the network generated by the active cooperation between experts in public health, infectious diseases, epidemiology and communications, and local institutions, as well as local doctors (general practitioners and paediatricians). These collaborate to serve the public in the promotion and protection of health, critical decision-making and conscious behaviour, and in tackling misinformation and fake news. VaccinarSinSardegna. org supports users in their individual and collective empowerment. In fact, the transfer of knowledge and skills shared on the portal, whose dedicated sections, periodic news and scientific publications are made available to all, aims to build each user's critical knowledge. The user is in turn an active part of an extended community (empowered organisation) in continuous growth, a harbinger of evidence and experience-based knowledge.

The aim for the future of the website is to grow further, expanding to reach an even wider audience and enjoy further collaborations with national and international scientific organisations and associations (29, 32, 33) . The authors conclude that initiatives such as this one should be implemented as part of an effective bid to counter the phenomenon of vaccine hesitancy and promote vaccination culture.

The main strength of the study is its ability to gauge users' need for scientifically reliable information even under pandemic conditions. Indeed, Vaccinarsinsardegna.org's ability to recognise and meet this need is favoured by the website's involvement in the Italian network VaccinarSì, (the medical and scientific communication sites on vaccinations in Italy) recognised and certified by the Health On the Net Foundation (HON), an accredited non-governmental organisation at the United Nations Economic and Social Council and WHO. Furthermore, the publication of evidencebased topics with a particular focus on local and regional information and the publication of the site also in English would seem to meet the needs of users by providing a punctual and effective service. The website's weaknesses may include a selection bias regarding the age of its users, indeed, the fact that social channels connected to the site have not yet been set up may negatively affect knowledge of the portal's existence among young audiences.

World Health Organization. World Immunization week

Vaccines for the21st century

The eradication of smallpox, a success story for modern medicine and public health: What lessons for the future?

Regional Reference Centres of the National Surveillance System for Acute flaccid paralysis. Hospital discharges-based search of acute flaccid paralysis cases 2007-2016 in Italy and comparison with the National Surveillance System for monitoring the risk of polio reintroduction

Impact of Influenza Vaccination on All-Cause Mortality and Hospitalization for Pneumonia in Adults and the Elderly with Diabetes: A Meta-Analysis of Observational Studies. Vaccines (Basel). 2020; 30

Is it time to reconsider measles, mumps, and rubella immunisation strategies?

Safety and perception: What are the greatest enemies of HPV vaccination programmes? Vaccine

Determinants of vaccine hesitancy and effectiveness of vaccination counseling interventions among a sample of the general population in Palermo

Flu Vaccination Attitudes, Behaviours, and Knowledge among Health Workers

Understanding modern-day vaccines: What you need to know

Social deprivation indexes and anti-influenza vaccination coverage in the elderly in Sardinia, Italy, with a focus on the Sassari municipality

Maintain and increase vaccination coverage in children, adolescents, adults and elderly people: Let's avoid adding epidemics to the pandemic: Appeal from the Board of the Vaccination Calendar for Life in Italy: Maintain and increase coverage also by re-organizing vaccination services and reassuring the population

The Experience of VaccinarSinToscana Website and the Role of New Media in Promoting Vaccination. Vaccines (Basel)

Information sources and knowledge on vaccination in a population from southern Italy: The ESCULAPIO project

Report of the SAGE Working Group on Vaccine Hesitancy

Vaccine hesitancy, vaccine refusal and the anti-vaccine movement: Influence, impact and implications

Vaccine hesitancy: Where we are and where we are going

Environmental Risks Perception Among Citizens Living Near Industrial Plants: A Cross-Sectional Study

Air pollutants and risk of death due to COVID-19 in Italy

In the Digital Era, Is Community Outrage a Feasible Proxy Indicator of Emotional Epidemiology? The Case of Meningococcal Disease in Sardinia

Chinese public attention to COVID-19 epidemic: Based on social media

mHealth: Use of Appropriate Digital Technologies for Public Health: Report by the Director-General. World Health Organization

The Effect of Information Communication Technology Interventions on Reducing Social Isolation in the Elderly: A Systematic Review

A Framework for Competencies for the Use of Mobile Technologies in Psychiatry and Medicine: Scoping Review

Public health digitalization in Europe: EUPHA vision, action and role in digital public health

The effectiveness of M-health technologies for improving health and health services: a systematic review protocol

Understanding and Preventing Health Concerns About Emerging Mobile Health Technologies

Current Status and Prospects for Information Communication Technology (ICT) in Allied Health Education

Improving vaccination social marketing by monitoring the web

Monitoring the web to support vaccine coverage: Results of two years of the portal VaccinarSì Epidemiol

Six years of activity of the Italian vaccine portal "VaccinarSì": a web traffic evaluation using Google Analytics

Health Communication in COVID-19 Era: Experiences from the Italian VaccinarSì Network Websites

Monitorare la Fiducia del Pubblico nei Programmi Vaccinali e le sue Necessità Informative Sviluppando un Sistema di Decisione Assistita per le Vaccinazioni Tramite il Sito "vaccinarsi.org" e Altri Siti e Social Network Specificatamente Dedicati Alle Vaccinazioni

What can internet users' behaviours reveal about the mental health impacts of the COVID-19 pandemic? A systematic review

A postmodern Pandora's box: anti-vaccination misinformation on the Internet

Identifying and addressing vaccine hesitancy

Countering vaccine hesitancy through immunization information systems, a narrative review

Can Digital Tools Be Used for Improving Immunization Programs?

Utilizing health information technology to improve vaccine communication and coverage

Guiding Principles for Immunization Activities during the COVID-19 Pandemic. Interim Guidance

COVID-19 and Living space challenge. Well-being and Public Health recommendations for a healthy, safe, and sustainable housing

Importance of Internet surveillance in public health emergency control and prevention: evidence from a digital epidemiologic study during avian influenza A H7N9 outbreaks

Pro-Environmental Behaviors: Determinants and Obstacles among Italian University Students

Environment and health: Risk perception and its determinants among Italian university students

Influenza Vaccination Strategies in Healthcare Workers: A Cohort Study

Delega al Governo in materia di sperimentazione clinica di medicinali nonche' disposizioni per il riordino delle professioni sanitarie e per la dirigenza sanitaria del Ministero della salute (GU Serie Generale n.25 del 31-01-2018)

The authors would like to thank Emma Dempsey for the English Language revision, and Luciano Murrone for the data management.

Conflict of Interest: the authors declare they have no commercial associations (e.g. consultancies, stock ownership, equity interest, patent/licensing arrangement etc.) that might pose a conflict of interest in connection with the submitted article.