key: cord-0284750-38vi0tb5
authors: Mills, M. C.; Ruettenauer, T.
title: The impact of mandatory COVID-19 certificates on vaccine uptake: Synthetic Control Modelling of Six Countries
date: 2021-10-11
journal: nan
DOI: 10.1101/2021.10.08.21264718
sha: 0d039c265b1eb12811e0684cebf2e799fdbdf38d
doc_id: 284750
cord_uid: 38vi0tb5

Background. COVID certification has been introduced, yet there are no empirical evaluations of its impact on vaccine uptake. Methods. Mirroring an RCT, we designed a synthetic control model comparing six countries (Denmark, Israel, Italy, France, Germany, Switzerland) that introduced certification (May-August 2021), with 20 control countries. Our estimates provide a counterfactual trend estimating what would have happened in virtually identical circumstances if certificates were not introduced. The primary outcome was daily COVID-19 vaccine doses, with supplementary analyses of COVID-19 infections. Findings. COVID-19 certification led to increased vaccinations 20 days prior to implementation, with a lasting effect up to 40 days after. Countries with lower than average pre-intervention uptake had a more pronounced increase. In France, doses exceeded 25,895 vaccines per million capita (pmc) or in absolute terms, 1,749,589 doses prior to certification and 11,434 pmc after (772,563 doses). There was no effect in countries with higher uptake (Germany) or when introduced during limited supply (Denmark). There was higher uptake for <20 years and 20-29 years. Access restrictions linked to certain settings (nightclubs, events >1,000) were associated with higher uptake <20 years. When extended to broader settings, uptake remained high in the youngest group, but also observed in older age groups. The relationship of the intervention with reported infections was difficult to assess based on available data. Interpretation. We provide the first empirical assessment of the relationship between COVID-19 certification and vaccine uptake. Interpretation should recognise additional factors, including age eligibility changes and pandemic trajectories. We provide evidence that certification could increase vaccine uptake. Funding and Competing Interest Statement. MCM receives funding from the Leverhulme Trust (Large Centre Grant), European Research Council (835079) and participates in UK SAGE SPI-B (behavioural insights) committee. The funders had no role in study design, data collection, analysis, interpretation, or writing of the report.

Countries have either introduced or are considering mandatory COVID-19 certification using proof of a double vaccination, negative PCR or lateral flow test, or a viral antibody serological test to demonstrate recent natural infection. 1,2 Certification manages entry into settings such as nightclubs, large events, hospitals, gyms or indoor hospitality (see Supplementary Material, Table A1 ). Numerous media reports have linked certification to increased vaccine uptake, 3-5 yet evidence remains anecdotal. Self-reported surveys suggest certification lowers vaccine intentions, 6 and a recent systematic review, 7 concluded that the quality and quantity of studies was low and concluded there was no evidence it would increase vaccine uptake. 7 Given that certain groups, such as youth, men and some ethnic minorities have lower levels of vaccine uptake -often attributed to vaccine complacency and lower perceptions of risk from COVID-19 8,9 -a relevant question is whether certification could be an additional policy lever to increase uptake. Vaccines provide not only protection for the immunised, but reduces transmission and risk of serious illness and death and only when sufficient numbers are vaccinated, can the chain of infection be broken. 10 Given mixed findings and lack of empirical evidence, we conducted the first empirical study to assess the relationship between the introduction of COVID-19 certification on observed vaccine uptake from May to 29 August 2021.

Information on COVID-19 related health indicators was used from Our World in Data, 11, 12 which harmonises daily information on indicators such as cases, deaths, and vaccinations. We linked these to the Oxford COVID-19 Government Response Tracker (OxCGRT), 13 which provides daily country information on the implementation and stringency of non-pharmaceutical interventions. Age-specific analyses used European Centre for Disease Prevention and Control (ECDC) data 14 on weekly agespecific vaccine doses. For all sources, we used the data around the time of COVID certification in each country in 2021 until 28 September 2021.

We selected six cases of Denmark, France, Germany, Israel, Italy and Switzerland, that implemented this intervention between May and August 2021, where sufficient data and time was available to examine effects. Details on when each intervention was announced, introduced, setting, rules or exemptions is in Supplementary Material Table A1 , with the timing shown in Figure 1 . A country was considered as implementing mandatory certificates if access to at least some frequently used public venues such as restaurants, or cultural events was restricted to people with a certificate (see Table A1 ). Some countries brought in certificates earlier, but did not associate restrictions or only applied rules to specific settings (e.g. Switzerland, larger events since 01 July). In the main analysis, we focus only on general access restrictions and examine different settings in a secondary analysis. In some cases, certificates were introduced gradually in different settings such as in Denmark in some settings on 6 April (e.g., hairdressers) when most public places were closed and later extended to most public places (21 April).

The primary outcome is daily new COVID-19 vaccination doses administered per capita, using a 7day smoothed rolling average. 12 For age-specific vaccination doses, we allocated the age-specific weekly doses (by ECDC) 14 is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 11, 2021. ; https://doi.org/10.1101/2021.10.08.21264718 doi: medRxiv preprint million). Harmonized age-specific vaccination rates from the ECDC were only available for France, Italy, and Denmark among the treated countries.

Age-specific vaccination data from the ECDC provides harmonized data, allowing us to run Synthetic Control models (see below) on age-specific vaccination rates. However, data is only available for relatively coarse age groups. We therefore compiled original data on detailed age-specific vaccination rates directly from national sources for France, 15 Israel, 16 Italy, 17 and Switzerland. 18 Since data from Israel came without age-specific counts of the total population, we supplemented this population information. 19 For Switzerland, the age-specific vaccination data is only available on a weekly basis. We therefore applied the same method as described above for the ECDC data.

We used a Synthetic Control method, which constructs a synthetic control group for each single treated country 20,21 by reweighting non-treated countries from the potential pool of control units in a way that the average pre-treatment trend (and other selected characteristics) closely mirroring the trend of the single treated unit. The synthetic control group average provides a counterfactual trend of the outcome for the treated unit. We estimate the effect of certificates on new daily vaccine doses.

Matching was performed on time-constant country characteristics: (1) median age, (2) percent 70 and older, (3) life expectancy, (4) GDP per capita; and, (5) population density and time-varying characteristics: (6) pre-treatment outcome (8-14, 15-21, 22-28 days before intervention); (7) stringency sub-indices of 12 NPIs (8-21 days before intervention), including: school closing, workplace closing, cancelled public events, restricted gatherings, closed public transport, stay at home, restricted internal movement, international travel, protecting elderly, testing policy, contact tracing, face coverings; (8) daily COVID-19 cases (1-7, 8-14, 15-21, 22-28 days before intervention); and, (9) percent fully vaccinated (8-21 days before intervention).

In a secondary analyses we examine COVID-19 reported infection cases as the outcome, including the following variables (instead of daily cases and percent fully vaccinated): (1) cumulative vaccinations per 100 inhabitants (1-7, 8-14, 15-21, 22-28 days before intervention); and, (2) daily tests per capita performed (8-21 days before intervention).

The following countries that did not introduce certification are included in the pool of control countries: Austria, Belgium, Canada, Czech Republic, Spain, Finland, Great Britain, Turkey, Croatia, Ireland, Lithuania, Luxembourg, Netherlands, Norway, Poland, Portugal, Slovakia, Slovenia, Sweden, and the United States. Due to data availability in some control countries, the following countries were included as control units for the age-specific analyses: Austria, Belgium, Cyprus, Czech Republic, Spain, Estonia, Finland, Turkey, Croatia, Ireland, Iceland, Lithuania, Luxembourg, Latvia, Malta, Norway, Poland, Portugal, Romania, Slovakia, Slovenia, and Sweden. We note that the results look similar for these two sets of control units. Analyses were performed using the R package gsynth, 22 and results visualized using ggplot. 23 In supplementary analyses, we used the Generalized Synthetic Control (GSC) method 24 to estimate an average effect across our selected cases in comparison with remaining countries included in Our World in Data. The GSC method uses the information of control units, and the pre-treatment trends of the treated units to impute the expected outcome of the treated units after receiving the intervention. 25 As covariates, we included the predictors mentioned above on a daily basis and two-ways fixed effects to account for country-specific differences and time-specific shocks.

We first assess the relationship between COVID-19 certification and overall vaccine uptake per capita. Figure is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 11, 2021. ; https://doi.org/10.1101/2021.10.08.21264718 doi: medRxiv preprint per capita compared to other similar countries until three weeks prior to the intervention. When mandatory certificates were announced in mid-July, daily vaccinations started to rise to an aboveaverage level and remained there after certificates were required.

We quantified the relative and absolute gains in vaccine uptake (Supplementary Table A5 .1) of the anticipation of introduction (sum over 20 days before) and afterwards (sum over 40 days after). The anticipation effect in France meant that vaccine uptake exceeded the control country by 25,895 vaccines per million capita or in absolute terms, 1,749,589 doses. Vaccine uptake up to 40 days after certification exceeded the control country by 11,434 vaccines per million capita or 772,563 doses. Our World in Data, 26 OxCGRT. 27 Control pool: see methods.

For Israel (Figure 3 ) we observe a similar shift before re-implementing the Green Pass. Three weeks before implementation, Israel reached the same level of vaccinations as the control group, providing evidence for an anticipation effect (although only towards average). The re-implementation coincided with a sharp increase in daily vaccinations directly afterwards and 40 days after implementation, rates were still more than 5,000 doses per million inhabitants higher. There was a limited anticipation effect, but a surge of 241,286 vaccines per million capita in total up to 40 days after which in absolute terms was 2,120,849 doses. We reflect on changes in eligibility by age groups in the next section.

For Italy (Figure 4) , we again observe a shift from a below-average rate of daily vaccinations until the announcement of the Green Pass, followed by a decrease slightly below average of other countries after implementation. At 30 days after implementation, daily doses in Italy were around 1,500 doses above the synthetic control group, again confirming a positive relationship between certification and vaccine uptake. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint Our World in Data, 26 OxCGRT. 27 Control pool: see methods. Data sources: Our World in Data, 26 OxCGRT. 27 Control pool: see methods.

Data in Switzerland ( Figure 5 ) shows that around a month before the introduction of certificates, they had lower vaccination levels than control countries and daily doses approach an above-average trend briefly before the intervention. The observation period is too short for conclusions about longer-term trends, but does confirm an anticipation effect. Supplementary results ( Figure A4 .1) based on different estimation methods confirm the long lasting upward shift after certification. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 11, 2021. ; https://doi.org/10.1101/2021.10.08.21264718 doi: medRxiv preprint Data sources: Our World in Data, 26 OxCGRT. 27 Control pool: see methods.

There are country-specific differences given different contexts and reasons for introducing certification, to increase testing (Denmark) or in response to rising cases (France, Germany, Belgium). In Denmark and Germany ( Figures A1.1, A1. 2), we do not observe an effect on vaccination rates. Vaccination rates in Denmark were above others and the goal was to encourage regular testing, with the certificate now discontinued from 10 September given high vaccination levels. For Germany, vaccination rates increased three weeks after the intervention, but absolute differences remain small. Denmark introduced certificates very early in 2021 in times of limited supply. Germany has a federal structure of decision making, which resulted in location-specific rules and exemptions from certification based on local incidence rates (see Table A1 ). Germany was also in the midst of a federal election, where the two leading candidates publicly opposed certification. 5 Both countries did not start from a below-average level of vaccine uptake compared to control countries before the intervention, indicating that the most leverage of certification was only if vaccination rates were below average than comparable countries.

We conducted secondary analyses on COVID-19 infections (Supplement A2) and concluded that the effect of certificates on reported cases is difficult to assess based on available data. For some countries (France, Germany, Italy) we observe a downwards trend after the intervention, whereas for others (Israel, Denmark) we observe a continued increase above the rate in other countries. The context of the pandemic trajectory when implementing the measure remains important (see Figure 1 ).

Countries had different age-specific eligibility rollouts and first versus second doses (see Supplementary Table A2 ). Given the settings where certification was introduced such as nightclubs and large events, some age groups may respond differently. In Figure 6 , we repeat previous analyses for France and Italy, but with age-specific vaccination rates. Both countries show a strong effect (anticipation and afterwards) for 25-49 year olds, while older age-groups seem less responsive, but also had a longer eligibility period to be vaccinated. In France, there is only a short period of increased uptake amongst the youngest ages after intervention. In Italy, the youngest age group (18) (19) (20) (21) (22) (23) (24) (25) had an increase directly around the intervention and an even sharper increase 2-3 weeks after certification.

. CC-BY-NC 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 11, 2021. ; https://doi.org/10.1101/2021.10.08.21264718 doi: medRxiv preprint

Data sources: Our World in Data, 26 OxCGRT, 27 European Centre for Disease Prevention and Control. 14 Control pool: see methods.

In addition to age-specific analyses based on synthetic control models, we also compiled age-specific data for France, Israel, Italy, and Switzerland (Figure 7) , allowing a more detailed descriptive account of age groups. This fine-grained analysis confirms that those below 20 and 20-29 years old had increased uptake.

One obvious concern is that our results are an artefact of age-related vaccine roll out and mirror the expanded eligibility to younger age groups during this period. To examine this, we link our findings to the timing of age-related eligibility (Supplementary Material Table A2 ). For France, children 12+ were eligible for vaccination on June 15, much earlier than the introduction of the certificate in August. In Italy, vaccines were available for children 12+ on August 16, which could partially explain the increase from mid-August, 2 weeks after certification was introduced. However, eligibility does not explain the pronounced increase amongst 20-29 year olds, eligible for a longer period. The two youngest age groups in Italy continued to have the highest vaccination rate amongst all age groups after the intervention. Findings hold for all doses, but also first dose only ( Figure A3 .1).

Although the sharp uptake amongst the youngest age groups in Israel could be attributable to second doses following the first spike, the intervention appears to be an uptake of first doses amongst teens below 20. Eligibility for vulnerable 5-11 year olds was introduced at the end of July, but this smaller group could not explain the sharp increase. Although children aged 12-15 were eligible on 06 June 2021, by the end of June only 2-4% of this age group were vaccinated. 28 After a surge in school infections, the Prime Minister urged citizens to vaccinate their children at the end of June, warning that the allotted doses would expire on July 9. Healthcare providers reported appointments for [12] [13] [14] [15] year olds tripled directly, suggesting increased infections and a public announcement of urgency and waste may have prompted vaccinations. 28 Supplementary analyses ( Figure A3.1) show an increase around the introduction of certificates (July 29) for first doses only, which seems independent of the earlier announcement. Although smaller, we find an anticipation effect and a long-lasting increase in uptake after the intervention for first doses in Israel.

In Switzerland, vaccines were approved for 12-15 year olds on 04 June 2021, considerably before the introduction of stricter certification in mid-September. The actual rollout depended on the Canton, is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 11, 2021. ; https://doi.org/10.1101/2021.10.08.21264718 doi: medRxiv preprint where for example, Zurich started to vaccinate younger ages on June 25 and Luzern only at the end of July. This increased rollout might have affected vaccine uptake of the first introduction of targeted certification rules (July 7), but it seems unlikely to affect our main intervention date (mid-September). It seems implausible that the staged introduction across Cantons is responsible for the small spike around the introduction of the first certification stage in early July. In a secondary analysis, we use a natural experiment where Switzerland first introduced more targeted access restrictions (events > 1,000 participants and nightclubs), and later extended them to general situations and activities (Table A1 ). In Figure 8 the dashed line represents the first targeted introduction and the solid line to more general public settings (events >30 participants, entire hospitality sector, leisure activities). The targeted access restrictions appear to have only affected the youngest (below 20), while we barely observe a change amongst older groups. Extending these restrictions to more general activities continued to impact uptake in the youngest, but also uptake amongst older age groups (30-39, 40-49).

. CC-BY-NC 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 11, 2021. ; https://doi.org/10.1101/2021.10.08.21264718 doi: medRxiv preprint 

Mandatory COVID-19 certification led to a sharp increase in vaccination rates prior to implementation and had a long-lasting effect with above-average rates after implementation. The impact was dependent on the average pre-intervention levels of vaccine uptake in the country and reasons for introducing certification, with those that had lower average levels of uptake showing more pronounced effects (France, Italy, Israel), particularly in some age groups. There was no obvious effect in countries that had a higher uptake (Germany) or in times of limited supply or when it was introduced to increase testing (Denmark). The effect of certificates on COVID-19 infections is difficult to assess, but it is logical that encouraging people who attend higher risk venues to be vaccinated would reduce transmission risk.

The age effect is related to a combination of settings of where certificates were introduced that disproportionately impacted youth and changes in the eligibility criteria. A detailed age-specific analysis revealed that particularly those below the age of 20 had the highest increased uptake, followed by those 20-29 years, holding for both first and second doses. When introduced and incentives in particular settings (nightclubs, events >1,000 people), higher uptake was found mainly in the youngest age group in Switzerland and when certification was extended to broader settings (events >30 people, entire hospitality sector, leisure activities) and increase was also observed in older groups (30) (31) (32) (33) (34) (35) (36) (37) (38) (39) (40) (41) (42) (43) (44) (45) (46) (47) (48) (49) .

A recent survey in the UK concluded that vaccine passports were unlikely to shift vaccination intentions in younger age groups. 6 Our results show the importance of moving away from selfreported intentions only to objective measures of behaviour. The current findings suggest that the introduction of COVID certification does increase vaccine uptake, particularly in younger age groups when linked to settings such as nightclubs and large events, and additionally in older age groups when applied to broader settings such as the entire hospitality sector. We also show that this intervention gains the most leverage if vaccination rates are below average. Our findings should be interpreted in . CC-BY-NC 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 11, 2021. ; https://doi.org/10.1101/2021.10.08.21264718 doi: medRxiv preprint relation to the national context of pre-existing levels of vaccine uptake, vaccine hesitancy, levels of trust in government and vaccinations, and pandemic trajectory. COVID certification is part of a constellation of multiple policy levers that could counter vaccine complacency and increase uptake. 8 Youth and certain groups such as men and those from low socioeconomic environments demonstrate higher levels of vaccine complacency due to a lower assessment of the risk of COVID- 19, 8, 9 suggesting certification could be one mechanism to increase uptake to reach vaccine immunity threshold levels to protect the broader population. Certification alone is not a silver bullet to increase vaccine uptake, with other measures such as geographically targeted interventions being more effective for certain groups. 30 Limitations include a lack of access to granular daily age-based uptake for all countries and ability to examine confounders such as ethnicity or socioeconomic status. Certification was introduced at different phases during the pandemic for different reasons and across regional and national-level conditions with varying levels of age eligibility, supply, vaccine hesitancy, enforcement and variation in infections and mortality. France 31 and Italy 32 have a history of vaccine hesitancy; 31 together with Israel incentivised vaccine uptake with the use of certification for desirable events or settings. 33 Whereas Belgium's aim was to avoid the reinstatement of restrictions. 34 Transferability of results to other settings should account for contextual aspects listed above.

Data sharing. All data is publicly available and listed in this document.

Contributors. MCM and TR conceived the project and designed the study. MCM wrote the first draft of the manuscript, edited by both authors. TR conducted the statistical analyses. MCM and TR engaged in policy review. Both authors had full access to the data and final responsibility to submit for publication.

The authors declare no competing interests or financial relationship with any organisation that might have an interest in the submitted work.

Mills MC, Dye C. Twelve criteria for the development and use of COVID-19 vaccine passports. R Soc 2021. https://royalsociety.org/-/media/policy/projects/set-c/set-c-vaccinepassports.pdf?la=en-GB&hash=A3319C914245F73795AB163AD15E9021. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 11, 2021. ; https://doi.org/10.1101/2021.10.08.21264718 doi: medRxiv preprint

COVID-19 vaccine hesitancy: the five Cs to tackle behavioural and sociodemographic factors

Awareness, Attitudes, and Actions Related to COVID-19 Among Adults With Chronic Conditions at the Onset of the U.S. Outbreak

Simply put: Vaccination saves lives

A global database of COVID-19 vaccinations

Coronavirus pandemic (COVID-19). Our World Data

A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker)

COVID-19 Vaccine rollout overview

Données relatives aux personnes vaccinées contre la Covid-19. VAC-SI Datasets

The ages of the vaccinated. data.gov.il

Commissario straordinario per l'emergenza Covid-19 -Presidenza del Consiglio dei Ministri. Covid-19 Opendata Vaccini. Covid-19 Opendata Vaccini

Federal Office of Public Health. COVID-19 Switzerland. COVID-19 Switz

World Population Prospects

Comparative Politics and the Synthetic Control Method

Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California's Tobacco Control Program

Synth : An R Package for Synthetic Control Methods in Comparative Case Studies

Elegant Graphics for Data Analysis

Generalized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models

Matrix Completion Methods for Causal Panel Data Models

COVID-19 Data by Our World in Data

OxCGRT Covid policy tracker data

School COVID-19 cases spur Israeli parents to vaccinte kids

Geographically targeted COVID-19 vaccination is more equitable and averts more deaths than age-based thresholds alone

Low Acceptability of A/H1N1 Pandemic Vaccination in French Adult Population: Did Public Health Policy Fuel Public Dissonance?

The law on compulsory vaccination in Italy: an update 2 years after the introduction

Incentivizing Vaccination Uptake

Risico dat regels anders meteen vernietigd worden" [Why coronapas is being introduced everywhere quickly and it just keeps on going here