key: cord-0940043-24idoiaz
authors: McMenamin, Martina E.; Bond, Helen S.; Sullivan, Sheena G.; Cowling, Benjamin J.
title: Estimation of Relative Vaccine Effectiveness in Influenza: A Systematic Review of Methodology
date: 2022-02-24
journal: Epidemiology
DOI: 10.1097/ede.0000000000001473
sha: 5ede238aea8cf2d543bc542909f2dbd71dc2c5a5
doc_id: 940043
cord_uid: 24idoiaz

BACKGROUND: When new vaccine components or platforms are developed, they will typically need to demonstrate noninferiority or superiority over existing products, resulting in the assessment of relative vaccine effectiveness (rVE). This review aims to identify how rVE evaluation is being performed in studies of influenza to inform a more standardized approach. METHODS: We conducted a systematic search on PubMed, Google Scholar, and Web of Science for studies reporting rVE comparing vaccine components, dose, or vaccination schedules. We screened titles, abstracts, full texts, and references to identify relevant articles. We extracted information on the study design, relative comparison made, and the definition and statistical approach used to estimate rVE in each study. RESULTS: We identified 63 articles assessing rVE in influenza virus. Studies compared multiple vaccine components (n = 38), two or more doses of the same vaccine (n = 17), or vaccination timing or history (n = 9). One study compared a range of vaccine components and doses. Nearly two-thirds of all studies controlled for age, and nearly half for comorbidities, region, and sex. Assessment of 12 studies presenting both absolute and relative effect estimates suggested proportionality in the effects, resulting in implications for the interpretation of rVE effects. CONCLUSIONS: Approaches to rVE evaluation in practice is highly varied, with improvements in reporting required in many cases. Extensive consideration of methodologic issues relating to rVE is needed, including the stability of estimates and the impact of confounding structure on the validity of rVE estimates.

V accines have greatly reduced the burden of many infectious diseases. Although randomized trials are required to evaluate a new vaccine, once that vaccine is approved and licensed it will be subject to postmarketing surveillance using observational studies. 1 A variety of observational study designs have been used to monitor vaccine effectiveness (VE). Estimation of causal effects such as VE 2,3 can be challenging in observational studies because of the potential for confounding. 4 The most commonly used study design for estimation of VE is the test-negative design (TND), in which a single clinical case definition is used for enrollment of participants and laboratory testing is subsequently employed to classify each patient into either the case or control group. [5] [6] [7] A recent review identified 348 articles using the TND for monitoring VE of 12 pathogens. 8 When new vaccine components or platforms are developed, they will typically need to demonstrate noninferiority or superiority over existing products. In these studies, the effectiveness of the new vaccine is to be compared with the existing vaccine to estimate the relative vaccine effectiveness (rVE). After licensure, other relative comparisons may also be of interest, such as VE by time since vaccination if there is a concern over waning VE 9, 10 ; VE by prior vaccination status 11, 12 ; VE by vaccine brand or platform 13 ; or VE by genetic clade or subgroup of the pathogen 14, 15 ; all of which will involve estimating a relative effect of one or more vaccines.

As the vaccine development landscape continues to advance, we expect to see an increased focus on relative vaccine comparisons. In particular, the National Institute for Allergy and Infectious Diseases (NIAID) has constructed a strategic plan to support research to develop new and improved vaccines for influenza. 16 Furthermore, as of 18 August 2021, 19 vaccines for coronavirus disease 2019 (COVID-19) have been licensed for either emergency or full use globally, [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] with a further 15 currently in phase 3 development, [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] including one that will be compared with the conditionally approved vaccine Vaxzevria, rather than placebo, 51 and one that will investigate the effect of the Comirnaty vaccine in those who have already received one dose of Vaxzevria. 52 Furthermore, studies with multiple relative comparisons are emerging which evaluate effectiveness with respect to prior infection, time since vaccination and different vaccine platforms. 53 In the coming months and years, assessment of the relative effectiveness of these vaccines as well as the comparative effect of individual vaccines against different emerging viral variants will be critical. Moreover, quantifying the relative effect of the vaccine at one time point versus another will be a priority for understanding waning in vaccine-derived immunity. Estimates of rVE therefore have a crucial role to play in policy making, most imminently for COVID-19 but also more routinely for seasonal pathogens such as influenza. Consequently, it is essential that rVE estimates are valid and procedures for obtaining these are standardized.

A first step to aid interpretation and make methodologic recommendations for future rVE estimates is identification of how relative vaccine comparisons are conducted in practice. Therefore, the objectives of this study are as follows: to review how rVE is evaluated for influenza vaccines; to identify common categories of rVE comparisons within influenza; to provide a summary of the different estimands and estimation techniques employed; to determine the confounding structures assumed; and to assess the bias and consistency in these estimates. We anticipate the findings will have implications for the ongoing assessment of vaccines for influenza, COVID-19, and other vaccine-preventable diseases, as well as relative comparisons of other interventions.

We followed the PRISMA guidelines when conducting this review and considered studies in any language. A systematic search was carried out on PubMed, Google Scholar, and Web of Science on August 18, 2021, using the following search term:

1. "vaccine" OR "vaccination" 2. "relative effectiveness" OR "relative efficacy" 3. #1 AND #2 4. "relative vaccine effectiveness" OR "relative vaccine efficacy" OR "relative VE" 5. #3 OR #4 6. "vaccine effectiveness" OR "VE" 7. #6 AND "waning" 8. #5 OR #7

We included the "waning" term in the search criteria as an additional option to allow for the inclusion of studies estimating effectiveness of a single vaccine at one time point relative to another that may not be identified using only terms #1 to #6. We also screened the reference lists of retrieved articles to identify any additional eligible studies.

We initially screened the articles identified in the search strategy to eliminate duplicates and then MMM and HB independently screened the remaining titles for relevance. We defined rVE studies as any providing estimates for the comparative effectiveness or efficacy of two or more vaccine components (e.g., egg-based versus cell-based vaccines), doses, or vaccination schedules directly. We included studies assessing rVE indirectly, for instance through a network meta-analysis, provided they estimated a relative effect.

We excluded studies that estimated rVE for pathogens other than influenza virus, those that focused on treatments rather than vaccines, cost-effectiveness of vaccines, comparative effect of vaccine uptake determinants, articles providing an overview of the research landscape, simulation studies and animal or immunogenicity studies. We excluded studies that only estimated absolute VE or assessed waning by estimating VE of the vaccine against an unvaccinated control group at different time points. We also excluded studies that did not conduct any inference or those that estimated rVE but did not mention the statistical methods used. We included secondary analyses and meta-analyses if the studies provided additional rVE estimates or novel methods for assessing rVE. We excluded any papers conducting interim analyses when the final analysis was available.

MMM and HB extracted data from included articles after the full text screening using a standardized form. We extracted information on season, the types of comparison made, study design, sample size, endpoint, rVE definition, and statistical models or methods used for estimation. We also recorded the rVE estimate, the variables that were adjusted for, or those used for matching or stratification, whether absolute VE estimates were reported, and if any multiple testing corrections were applied (if applicable).

We classified vaccine comparisons according to three main categories: component, dose, and timing or history. This accounted for comparing different vaccine components, the same components with different brand names, the same vaccine with additional booster dose, the same vaccine across a dosing schedule (i.e., the time between vaccinations or cases in which multiple vaccinations are required within in a season), vaccination history, or whether VE was compared in two different time intervals to assess waning; all of which were considered relative effects. MMM and HB carried out a bias assessment using the "risk of bias in nonrandomized studies" (ROBINS-I) tool for observational studies, to assess the risk of bias classified as "low," "moderate," "serious," "critical," or "no information" across seven domains. 54 They assessed randomized studies using the "risk of bias in randomized trials" (ROB-2) tool, where bias risk was classified as "low," "some concerns," or "high" across five domains. 55 

Our search on PubMed, Google Scholar, and Web of Science (WOS) resulted in 356 articles in the first instance. We removed six duplicates and identified five additional relevant publications from screening the reference lists of published articles. After screening, we identified 63 articles that met criteria for inclusion ( Figure 1 ). Relative VE was estimated for either differing vaccine components (n = 38), dosing schedules (n = 17), or vaccination timing or history (n = 9), with one study assessing a range of components and doses, as detailed in the Table. We found 86% (n = 54) of relative VE studies have been conducted in the last decade, with 49% (n = 31) published in the last 2.5 years. FIGURE 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram for studies identified from a PubMed search with search term: ((("vaccine") OR ("vaccination")) AND (("relative effectiveness") OR ("relative efficacy"))) OR (("relative vaccine effectiveness") OR ("relative vaccine efficacy") OR ("relative VE")) OR ((("vaccine effectiveness") OR ("VE")) AND ("waning")), which were sorted as indicated. 

We found a range of study designs were used to compare relative effectiveness including retrospective cohort (n = 25), randomized controlled trial (RCT) (n = 22), systematic review and meta-analysis (n = 7), 56, 71, 75, 84, 91, 103, 111 TND (n = 6), 61, 64, 67, 76, 80, 101 case-control (n = 2), 69, 85 and prospective cohort (n = 1). 99 It is important to note that some of the rVE studies included in the review are also incorporated in the seven meta-analyzed estimates. This is owing to the primary aim being to identify rVE methods, including pooled estimands, rather than to draw conclusions on the interventions themselves.

Relative vaccine effectiveness was reported as a percentage in the majority of studies (n = 52) and was otherwise reported as a ratio (n = 11), as shown in Figure 2 . The most commonly used definition of relative vaccine effectiveness was (1 − IRR) x100 (n = 27) where IRR denotes the incidence rate ratio for one group versus the other. Other studies reported rVE using only the rate ratio (n = 4), 82, 88, 99, 110 prior event rate ratio (PERR) (n = 3) 62,63,78 or instrumental variable-adjusted (IVadj) rate ratio (n = 1). 70 The cluster-randomized trial estimated rVE by estimating the IRR of hospitalization for influenza and pneumonia in residents randomized at the facility level (n = 1). 82 Other definitions included (1 − HR) x100 (n = 7), 77, 93, 95, 97, 115, 117, 118 (1 − OR) x100 (n = 10), [57] [58] [59] 61, 64, 67, 69, 76, 101, 114 OR (n = 5), 65, 74, 80, 85, 94 and 1 OR x100 (n = 1), 68 where HR is the hazard ratio and OR is the odds ratio. The majority of systematic reviews and meta-analyses reported rVE as a meta-analyzed summary measure, including (1 − RR pool ) x100 (n = 1) 84 ; (1 − OR pool ) x100 (n = 1) 56 ; OR pool (n = 1) 71 and pooled change in VE (n = 1). 75 One study reported a pooled estimate combining ORs and IRRs. 111 Two studies provided a reference to justify the definition used. 112, 116 However, the cited publication had not provided any justification. 72 Overall, approximately one-third (n = 18, 29%) of the included studies reported absolute VE in addition to the relative vaccine effectiveness (see Figure 3) . A total of 83% (n = 5) of test-negative studies reported the absolute VE, compared with only 4% (n = 1) 59 of retrospective cohort studies.

Age was the most commonly controlled variable (n = 41, 65%) as both vaccination status and risk of infection change with age. Comorbidities (n = 29, 46%), geographic region (n = 29, 46%), and sex (n = 25, 40%) were also commonly accounted for. Ethnicity (n = 16, 25%), healthcare seeking proxies (n = 13, 21%), calendar time (n = 13, 21%), previous vaccination status (n = 11, 17%), and time of vaccination (n = 8, 13%) were among the other characteristics considered in analyses. These variables were either specified as a covariate in the regression model or were included in the calculation of propensity score matching. However, approximately onethird (n = 18, 29%) of studies either did not include or did not report the covariates considered in analyses.

The estimation of the rate ratio, OR, and HR estimands use a number of different statistical models and techniques. For estimators based on the rate ratio, models used included inverse probability weighted Poisson regression (n = 7), 60, 72, 79, 88, 112, 113, 116 standard Poisson regression (n = 3), 82,100,110 and nested Poisson regression models (n = 1). 66 Other approaches included Cox proportional hazards (n = 2), 89, 96 Andersen-Gill (n = 3), 103,105,106 log-binomial models (n = 1), 91 logistic regression (n = 1), 83 and generalized estimating equations (n = 1). 99 In studies comparing observed proportions in each group, Fishers exact test was used (n = 6) 73,90,102,104,108,109 and confidence intervals were constructed using the method of Clopper-Pearson (n = 3), 87, 92, 98 Blackwelder (n = 1), 86 Farrington-Manning (n = 1), 81 and Guess et al (n = 1). 119 Studies estimating the PERR used a Poisson regression including an interaction term between period and treatment (n = 3), 62,63,88 and multivariable instrumental variable Poisson regression was used to estimate the IV-adjusted IRR (n = 1). 70 Meta-analyses estimating pooled effects used a random effects model (n = 2), 71, 75 with DerSimonian-Laird estimators for the OR (n = 2), 56, 111 or a log-binomial model with study included as a fixed effect (n = 1). 84 Studies basing rVE estimates on the HR employed Cox proportional hazards (n = 4) 93,95,97,118 with treatment as main effect and stratifying covariates as random effects (n = 2) 77,115 or Fine-Gray subdistribution hazard models (n = 1). 117 For OR estimators, rVE was estimated using logistic regression (n = 10), 58 65, 69, 74 inverse probability weighted logistic regression (n = 1), 114 Cox regression (n = 1), 59 and doubly robust inverse probability of treatment weights (n = 1). 57 The incidence rates across nursing Note that some studies have more than one type of comparison. IIV indicates inactivated influenza vaccine; LAIV, live attenuated inactivated vaccine; TIV, trivalent inactivated vaccine.

homes were modeled using marginal Poisson regression with log of resident days as an offset term (n = 1). 82

We assessed 34 observational studies using the ROBINS-I tool. Overall, we classified 65% (n = 22) studies to be at "moderate" risk of bias, 18% (n = 6) 60,67,72,88,110,117 as having "serious" risk, and the remaining 18% (n = 6) 59,63,64,74,93,118 did not provide sufficient information to assess. The individual domain assessments are shown in Figure 4 . We deemed all studies to be at low risk of bias with respect to deviations from the planned interventions, and all studies were either at low or moderate risk of bias due to selection, classification of intervention, or reporting of results. We deemed one study to be at serious risk of bias with respect to confounding and measurement of outcomes. 110 Bias due to missing data was most common with over a quarter of studies (n = 9) 59, 63, 64, 68, 74, 79, 93, 110, 118 not providing sufficient information and we classified bias in 15% (n = 5) 60, 67, 72, 88, 117 of studies as serious.

We assessed 19 of the 22 RCTs included using the ROB-2 tool, as four publications related to different subgroup results from the same overall trial. 83, 86, 87, 92 We found 26% of studies (n = 5) 73, 77, 98, 108, 115 were at low risk of bias, deemed 47% of studies (n = 9) 82, 90, 96, 100, [104] [105] [106] [107] 109 to have some concerns of bias, and classified the remaining 26% (n = 5) 81, 83, 95, 97, 102 studies as at high risk of bias overall. The measurement of outcome domain was at the highest risk of bias with three studies being classified as high risk. 81, 83, 95 One study was classified as high risk with respect to deviations from the intervention and missing data. 97 All domains had studies with some concerns of bias relating to deviations from intended interventions (n = 7) 96, 102, [104] [105] [106] [107] 109 ; concerns about the randomization (n = 6) 82,90,100,102,105,106 ; missing data (n = 3) 90,95,107 ; measurement of outcomes (n = 2) 100,102 ; and reporting (n = 1). 82

In the subset of studies presenting both aVE and rVE, 12 studies reported aVE for both vaccines. 59, 61, 64, 67, 71, 76, 100, 101, 104, 108, 109, 115 Figure 5 shows the aVE of one vaccine versus aVE of the other vaccine included in the relative comparison, with the majority of studies supporting the hypothesis of proportionality in the vaccine effects. The practical consequences of this can be shown via two scenarios. In scenario 1, assume 3 subjects vaccinated with vaccine A, 5 subjects vaccinated with vaccine B, and 13 subjects unvaccinated, out of a total of 36 subjects. This results in an aVE of vaccine A of 76.9%, an aVE of vaccine B of 61.5%, and an rVE of B versus A of 40%. In scenario 2, assuming 12 subjects are vaccinated with vaccine A, 20 subjects vaccinated with vaccine B, and 23 subjects unvaccinated. This would result in an aVE of vaccine A of 47.8% and an aVE of vaccine B of 13%, which also translates to an rVE of 40%.

We identified and reviewed the methodology used to estimate rVE in 63 influenza VE studies. Over half of the identified studies compared the relative effect of two or more vaccine components, one-third focused on comparing doses or dosing schedules of one vaccine, and the remainder estimated the relative effect of vaccine timing or vaccination history. The majority of studies reported relative VE as a percentage. However, we observed substantial variation in the definitions and approaches employed, often with no justification provided for the chosen approach. This reflects the fact that little methodologic consideration has been given to the topic, resulting in a lack of available recommendations, which investigators can use to inform their analysis.

Across all study designs, the majority of rVE estimators were based on either a rate ratio, OR, or HR. Extensions of standard models were used in some studies to address limitations. For example, six studies used inverse probability treatment weighting to address potential biases resulting from issues with confounding and missing data. However, it is worth noting that these methods may not always outperform standard multivariable analysis in dealing with confounding and only balance with respect to observed rather than unobserved covariates. 120, 121 One study performed an instrumental variable-adjusted analysis, which aims to estimate a causal effect even if all confounding variables have not been measured and accounted for in the analysis. 122, 123 This could be a useful technique for estimating rVE, as the confounding structure is typically not as well understood and problems with self-selection in vaccination can be addressed. 124 However, identifying an instrumental variable, particularly in observational studies, remains challenging hence limiting its application. 125, 126 The use of Cox proportional hazards models was most common for estimation of rVE based on HRs. To obtain valid rVE estimates, it is important for investigators to assess the assumption of proportional hazards in their context. If it is unlikely to hold, we suggest an extension of the model, such as the Andersen-Gill method, which allows for multiple events and time-varying covariates. 127 This approach may also be preferred for smaller sample sizes, as including only the time of the first event would result in more imprecise effect estimates. In most cases, the best choice of model and framework will be study-dependent and factors such as the population of interest should be carefully considered at the design stage. A summary of the potential considerations is provided in eTable 1 (http://links. lww.com/EDE/B907) in the eAppendix (http://links.lww.com/ EDE/B907). Despite many study-dependent considerations, it remains important to standardize the definitions of rVE and reporting of rVE studies, to aid future comparisons and metaanalyses. Furthermore, it is important to ensure considerations relating to confounding structure and other potential biases in the assessment of rVE has been identified and investigated to identify shortfalls in existing methodology and approaches.

We found that only 29% of studies measured or reported a comparison with a nonvaccinated group, that is, by either conducting a primary estimation of absolute VE or reporting the assumed VE estimates, resulting in 71% of studies reporting relative effectiveness only. The omission of a comparison with an unvaccinated group has some important implications for interpretation, which were not highlighted or discussed in the majority of papers. Importantly, relative effect estimates tell us nothing about the absolute effectiveness of each individual vaccine and therefore whether the effect will translate to an impact on public health and policy. For example, two scenarios highlighted in the text result in an rVE of 40%; in scenario one the aVE is 61.5% versus 76.9% as opposed to scenario two, which is 13.1% versus 47.8%. Clearly these two scenarios would have differing effects on public health and so it is vital for policy makers to have both absolute and relative effect estimates. However, in reality, it is likely that many studies cannot feasibly incorporate an unvaccinated control group. For example, the unvaccinated group used in COVID-19 vaccine studies will change considerably over time and will be confounded by prior infection, which may be poorly documented. Hence, where absolute effectiveness cannot be estimated, authors should summarize and report the assumed individual efficacies obtained from previous studies in similar populations to provide the necessary context for the reader.

The aim of VE studies is to derive an estimate of a causal effect, not merely an association. 3 When done so in an observational setting, it is well established that obtaining valid estimates of causal effects requires identifying and controlling for confounding variables, such as age and comorbidities, which can be associated with both vaccination status and risk of infection. However, in the case of relative comparisons between two vaccinated groups, the confounding structure is not as well understood. This is reflected in our findings in that only 65% of studies adjusted for age compared with 97% of studies assessing VE with an unvaccinated group as the comparison. 8 We suggest that accounting for demographics, comorbidities, and other factors relevant to absolute VE estimation is still crucial to ascertain valid estimates of relative effectiveness, as it is plausible that factors, such as age, may still be associated with both the exposure and outcome. For instance, if the comparison of interest is the relative effect of influenza vaccination early versus late in the season in preventing influenza-like illness, then age is likely to be associated with both given that older people are prioritized to receive vaccination early in the season. If a substantial proportion of the 29% of studies not reporting adjustment for covariates did not consider confounding factors, then this may have resulted in substantial biases in the existing literature for relative vaccine effects. However, our bias assessment shows only one study was at serious risk of bias due to confounding. 110 Further consideration of the importance of confounding on the validity of causal interpretations of relative vaccine estimates is warranted.

Although we aimed to identify all possible comparisons relating to VE through our specified search criteria, it is possible due to the nonstandardized terminology for assessing rVE that we have missed the inclusion of some studies. One possible subset of studies not identified is the relative effect of a single vaccine on one genetic subgroup versus another. We expect this category of comparison to increasingly feature in the rVE literature, as the effectiveness of emerging vaccines on one SARS-CoV-2 variant versus another will be important to establish. In addition, as the review focused on rVE methods used in practice, we did not include in the Results those methods proposed in simulation or model-based studies. By considering the relevant simulation studies, we identified an additional 10 studies with substantial variability in rVE estimands used. It will be important to incorporate these more novel proposals in future comparisons of statistical properties of methods for rVE.

This review highlighted the lack of consideration given to methodologic and practical implications of relative vaccine effectiveness estimation within the literature. Based on our findings, we recommend better reporting of rVE studies to include the definition of rVE used, absolute VE either estimated within the study or assumed VE stated, discussion of confounding structure and what confounders will be included in the model along with how they are accounted for (i.e., adjusted, matched), and where relevant, multiple testing corrections were included and discussed. Extensive consideration of methodologic issues relating to rVE is needed, including estimand used and the impact of confounding structure on the validity and stability of rVE estimates. These issues should be investigated theoretically and empirically to improve the quality of evidence on rVE.

What makes placebo-controlled trials unethical?

The causal interpretation of "Overall Vaccine Effectiveness" in test-negative studies

Crude Vaccine Effectiveness" is a misleading term in test-negative studies of influenza vaccine effectiveness

Evidence of bias in estimates of influenza vaccine effectiveness in seniors

The test-negative design for estimating influenza vaccine effectiveness

Theoretical basis of the test-negative study design for assessment of influenza vaccine effectiveness

Observational studies and the difficult quest for causality: lessons from vaccine effectiveness and impact studies

The use of test-negative controls to monitor vaccine effectiveness: a systematic review of methodology

I-MOVE multicentre case-control study 2010/11 to 2014/15: is there within-season waning of influenza type/subtype vaccine effectiveness with increasing time since vaccination?

Waning vaccine effectiveness against influenza-associated hospitalizations among adults

Effects of prior season vaccination on current season vaccine effectiveness in the United States flu vaccine effectiveness network

Repeated annual influenza vaccination and vaccine effectiveness: review of evidence

DRIVE Public Partners. Brand-specific influenza vaccine effectiveness estimates during 2019/20 season in Europe-results from the DRIVE EU study platform

Clinical effectiveness of split-virion versus subunit trivalent influenza vaccines in older adults

Vaccine effectiveness against lineage-matched and -mismatched influenza B viruses across 8 seasons in Canada

A Universal influenza vaccine: the strategic plan for the National Institute of Allergy and Infectious Diseases

Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK

COVE Study Group. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine

Effect of an inactivated vaccine against SARS-CoV-2 on safety and immunogenicity outcomes: interim analysis of 2 Randomized Clinical Trials

Neutralization of UK-variant VUI-202012/01 with COVAXIN vaccinated human serum. bioRxiv

A Phase III Clinical Trial to Determine the Safety and Efficacy of ZF2001 for Prevention of COVID-19

Efficacy and Safety of QazCovid-in COVID-19 Vaccine

A Study to Evaluate the Efficacy, Safety and Immunogenicity of SARS-CoV-2 Vaccine (Vero Cells), Inactivated in Healthy Adults Aged 18 Years and Older (COVID-19)

COVIRAN Vaccine Starts Third Phase of Human Trial

Safety and immunogenicity of CpG 1018 and aluminium hydroxide-adjuvanted SARS-CoV-2 S-2P protein vaccine MVC-COV1901: interim results of a large-scale, double-blind, randomised, placebo-controlled phase 2 trial in Taiwan

Clinical Study-Phase III

Soberana02: Report of Clinical Efficacy Against Symptomatic Disease COVID-19 in a Schedule of Two Doses Every 28

C4591001 Clinical Trial Group. Safety and efficacy of the BNT162b2 mRNA covid-19 vaccine

Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia

Double-blind, randomized, placebo-controlled phase III clinical trial to evaluate the efficacy and safety of treating healthcare professionals with the adsorbed COVID-19 (inactivated) vaccine manufactured by Sinovac-PROFISCOV: a structured summary of a study protocol for a randomised controlled trial

Safety, tolerability, and immunogenicity of a recombinant adenovirus type-5 vectored COVID-19 vaccine: a doseescalation, open-label, non-randomised, first-in-human trial

Study of the Safety, Reactogenicity and Immunogenicity of "EpiVacCorona" Vaccine for the Prevention of COVID-19 (EpiVacCorona). ClinicalTrials.gov

Safety and immunogenicity of an inactivated SARS-CoV-2 vaccine, BBIBP-CorV: a randomised, double-blind, placebo-controlled, phase ½ trial

A Study of Ad26.COV2.S for the Prevention of SARS-CoV-2-Mediated COVID-19 in Adult Participants (ENSEMBLE)

Chumakov Federal Scientific Center for Research and Development of Immune-and-Biological Products of Russian Academy of Sciences. Third COVID-19 Vaccine

Evaluation of the Safety and Immunogenicity of a SARS-CoV-2 rS Nanoparticle Vaccine With/Without Matrix-M Adjuvant

Zydus Starts Human Dosing of its Vaccine "ZyCoV-D

Chinese Academy of Medical Sciences. The Efficacy, Safety and Immunogenicity Study of Inactivated SARS-CoV-2 Vaccine for Preventing Against COVID-19

A Phase III Clinical Study of a SARS-CoV-2 Messenger Ribonucleic Acid (mRNA) Vaccine Candidate Against COVID-19 in Population Aged 18 Years and Above

A Global Phase III Clinical Trial of Recombinant COVID-19 Vaccines (Sf9 Cells)

Home-grown Nano Covax Vaccine Enters Third-Phase Trial

Study of Monovalent and Bivalent Recombinant Protein Vaccines Against COVID-19 in Adults 18 Years of Age and Older (VAT00008)

A Study to Evaluate the Safety and Immunogenicity of Vaccine CVnCoV in Healthy Adults in Germany for COVID-19

BCG Vaccination to Protect Healthcare Workers Against COVID-19 (BRACE)

Phase II / III Study of COVID-19 DNA Vaccine (AG0302-COVID19)

Study of GRAd-COV2 for the Prevention of COVID-19 in Adults (COVITAR)

Cuba's Soberana 02 SARS-CoV-2 Vaccine Candidate Moves to Phase III Trials

Study of a Recombinant Coronavirus-Like Particle COVID-19 Vaccine in Adults

A Controlled Phase 2/3 Study of Adjuvanted Recombinant SARS-CoV-2 Trimeric S-protein Vaccine (SCB-2019) for the Prevention of COVID-19

Limited Gets CDSCO NOD to Start Phase III Clinical Trial of its COVID-19 Vaccine Candidate

Initiates Phase 3 Clinical Trial for its Inactivated, Adjuvanted COVID-19 Vaccine Candidate

Spanish Clinical Research Network-SCReN. Vaccination With COMIRNATY in Subjects With a VAXZEVRIA First Dose (CombiVacS). 2021. Available at

Impact of vaccination on SARS-CoV-2 cases in the community: a population-based study using the UK's COVID-19 Infection Survey. medRxiv

ROBINS-I: a tool for assessing risk of bias in nonrandomised studies of interventions

RoB 2: a revised tool for assessing risk of bias in randomised trials

Efficacy and effectiveness of high-dose influenza vaccine in older adults by circulating strain and antigenic match: an updated systematic review and meta-analysis

Relative effectiveness of adjuvanted trivalent inactivated influenza vaccine versus egg-derived quadrivalent inactivated influenza vaccines and high-dose trivalent influenza vaccine in preventing influenza-related medical encounters in US adults ≥ 65 year

Relative effectiveness of the cell-cultured quadrivalent influenza vaccine compared to standard, egg-derived quadrivalent influenza vaccines in preventing influenza-like illness in 2017

Vaccine effectiveness of cell-culture relative to egg-based inactivated influenza vaccine during the 2017-18 influenza season

Relative effectiveness of influenza vaccines among the United States elderly

Relative and absolute effectiveness of high-dose and standard-dose influenza vaccine against influenzarelated hospitalization among older adults-United States

Assessing the prior event rate ratio method via probabilistic bias analysis on a Bayesian network

Comparative effectiveness of high dose versus adjuvanted influenza vaccine: a retrospective cohort study

Comparison of vaccine effectiveness against influenza hospitalization of cell-based and egg-based influenza vaccines

Depletionof-susceptibles bias in analyses of intra-season waning of influenza vaccine effectiveness

Effect of age on relative effectiveness of high-dose versus standard-dose influenza vaccines among US medicare beneficiaries aged ≥65 years

Comparing influenza vaccine effectiveness between cell-derived and egg-derived vaccines, 2017-2018 influenza season

Influenza vaccine failure in the tropics: a retrospective cohort study of waning effectiveness

Adjuvanted versus nonadjuvanted influenza vaccines and risk of hospitalizations for pneumonia and cerebro/ cardiovascular events in the elderly

Analysis of relative effectiveness of high-dose versus standard-dose influenza vaccines using an instrumental variable method

Influenza Clinical Investigation for Children Study Team; Influenza Incidence Surveillance Project; US Influenza Vaccine Effectiveness Network. Live attenuated and inactivated influenza vaccine effectiveness

Relative effectiveness of cellcultured and egg-based influenza vaccines among elderly persons in the United States

Semiannual versus annual influenza vaccination in older adults in the tropics: an observer-blind, activecomparator-controlled, randomized superiority trial

Intraseason waning of influenza vaccine effectiveness

The impact of repeated vaccination on influenza vaccine effectiveness: a systematic review and metaanalysis

Effectiveness of live attenuated vs inactivated influenza vaccines in children during the 2012-2013 through 2015-2016 influenza seasons in Alberta, Canada: a Canadian Immunization Research Network (CIRN) study

Efficacy, immunogenicity, and safety evaluation of an MF59-adjuvanted quadrivalent influenza virus vaccine compared with non-adjuvanted influenza vaccine in children: a multicentre, randomised controlled, observer-blinded, phase 3 trial

Relative vaccine effectiveness of high-dose versus standard-dose influenza vaccines among veterans health administration patients

Comparative effectiveness of influenza vaccines among U.S. Medicare beneficiaries ages 65 years and older during the 2019-20 season

Valencia Hospital Network for the Study of Influenza and other Respiratory Viruses (VAHNSI, Spain)

PSC12 Study Team. Efficacy of recombinant influenza vaccine in adults 50 years of age or older

Comparative effectiveness of high-dose versus standard-dose influenza vaccination on numbers of US nursing home residents admitted to hospital: a cluster-randomised trial

Effect of previous-year vaccination on the efficacy, immunogenicity, and safety of high-dose inactivated influenza vaccine in older adults

A systematic review of the efficacy of live attenuated influenza vaccine upon revaccination of children

Seasonal effectiveness of live attenuated and inactivated influenza vaccine

Prevention of serious events in adults 65 years of age or older: a comparison between high-dose and standard-dose inactivated influenza vaccines

Efficacy and immunogenicity of high-dose influenza vaccine in older adults by age, comorbidities, and frailty

Centers for Disease Control and Prevention Epicenter Program. Comparative effectiveness of high-dose versus standard-dose influenza vaccination in community-dwelling veterans

Comparative effectiveness of high-dose versus standard-dose influenza vaccines in US residents aged 65 years and older from 2012 to 2013 using Medicare data: a retrospective cohort analysis

Efficacy, immunogenicity, and safety of a plant-derived, quadrivalent, virus-like particle influenza vaccine in adults (18-64 years) and older adults (≥65 years): two multicentre, randomised phase 3 trials

Efficacy of live attenuated influenza vaccine against influenza illness in children as a function of illness severity

Efficacy of high-dose versus standard-dose influenza vaccine in older adults

Health-related behaviors and effectiveness of trivalent inactivated versus live attenuated influenza vaccine in preventing influenza-like illness among young adults

Waning vaccine protection against influenza A (H3N2) illness in children and older adults during a single season

Influenza symptoms and their impact on elderly adults: randomised trial of AS03-adjuvanted or nonadjuvanted inactivated trivalent seasonal influenza vaccines. Influenza Other Respir Viruses

Relative efficacy of AS03-adjuvanted pandemic influenza A(H1N1) vaccine in children: results of a controlled, randomized efficacy trial

Influence65 study group. AS03-adjuvanted versus non-adjuvanted inactivated trivalent influenza vaccine against seasonal influenza in elderly people: a phase 3 randomised trial

High-dose trivalent influenza vaccine compared to standard dose vaccine in elderly adults: safety, immunogenicity and relative efficacy during the 2009-2010 season

Effectiveness of adjuvanted influenza vaccination in elderly subjects in northern Italy

Oil-in-water emulsion adjuvant with influenza vaccine in young children

US Flu VE Network Investigators. Relative effectiveness of high dose versus standard dose influenza vaccines in older adult outpatients over four seasons

A prospective, randomized, open-label trial comparing the safety and efficacy of trivalent live attenuated and inactivated influenza vaccines in adults 60 years of age and older

The efficacy of live attenuated and inactivated influenza vaccines in children as a function of time postvaccination

Comparative efficacy of inactivated and live attenuated influenza vaccines

Superior relative efficacy of live attenuated influenza vaccine compared with inactivated influenza vaccine in young children with recurrent respiratory tract infections

CAIV-T Asthma Study Group. Comparison of the efficacy and safety of live attenuated cold-adapted influenza vaccine, trivalent, with trivalent inactivated influenza virus vaccine in children and adolescents with asthma

CAIV-T Comparative Efficacy Study Group. Live attenuated versus inactivated influenza vaccine in infants and young children

Prevention of antigenically drifted influenza by inactivated and live attenuated vaccines

Prevention of symptomatic seasonal influenza in 2005-2006 by inactivated and live attenuated vaccines

Comparison of the trivalent live attenuated vs. inactivated influenza vaccines among U.S. military service members

Effectiveness of the MF59-adjuvanted trivalent or quadrivalent seasonal influenza vaccine among adults 65 years of age or older, a systematic review and meta-analysis. Influenza Other Respir Viruses

Evaluating the relative vaccine effectiveness of adjuvanted trivalent influenza vaccine compared to highdose trivalent and other egg-based influenza vaccines among older adults in the US during the 2017-2018 influenza season. Vaccines (Basel)

A retrospective cohort study assessing relative effectiveness of adjuvanted versus high-dose trivalent influenza vaccines among older adults in the United States during the 2018-19 influenza season

Effectiveness of the cell-derived inactivated quadrivalent influenza vaccine in individuals at high risk of influenza complications in the 2018-2019 United States influenza season

Efficacy of live attenuated and inactivated influenza vaccines among children in rural India: a 2-year, randomized, triple-blind, placebo-controlled trial

A real-world study evaluating the relative vaccine effectiveness of a cell-based quadrivalent influenza vaccine compared to egg-based quadrivalent influenza vaccine in the US during the 2017-18 influenza season

Relative vaccine efficacy of high-dose versus standard-dose influenza vaccines in preventing probable influenza in a Medicare Fee-for-Service population

High-dose influenza vaccination and mortality among predominantly male, white, senior veterans, United States, 2012/13 to 2014/15

Exact Binomial confidence intervals for the relative risk in follow-up studies with sparsely stratified incidence density data

Invited commentary: propensity scores

A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods

Identification of causal effects using instrumental variables

Instrumental variable analysis for estimation of treatment effects with dichotomous outcomes

The instrumental variable method to study selfselection mechanism: a case of influenza vaccination

Problems with instrumental variables estimation when the correlation between the instruments and the endogeneous explanatory variable is weak

Instrumental variables in influenza vaccination studies: mission impossible?! Value Health

Cox's regression model for counting processes: a large sample study

The authors thank Julie Au for administrative support.