key: cord-0989296-z1fat73y
authors: Meggiolaro, Angela; Sane Schepisi, Monica; Nikolaidis, Georgios F.; Mipatrini, Daniele; Siddu, Andrea; Rezza, Giovanni
title: Effectiveness of Vaccination against SARS-CoV-2 Infection in the Pre-Delta Era: A Systematic Review and Meta-Analysis
date: 2022-01-21
journal: Vaccines (Basel)
DOI: 10.3390/vaccines10020157
sha: dd480d51ce60bc21749659d1e1f74d0b4505e176
doc_id: 989296
cord_uid: z1fat73y

(1) Background: The objective of this study was to assess the effectiveness of SARS-CoV-2 vaccines in terms of prevention of disease and transmission in the pre-Delta era. The evaluation was narrowed to two mRNA vaccines and two modified adenovirus-vectored vaccines. (2) Methods: The overall risk of any SARS-CoV-2 infection confirmed by positive real-time Polymerase Chain Reaction (PCR) test was estimated in partially and fully vaccinated individuals. The evidence synthesis was pursued through a random-effects meta-analysis. The effect size was expressed as relative risk (RR) and RRR (RR reduction) of SARS-CoV-2 infection following vaccination. Heterogeneity was investigated through a between-study heterogeneity analysis and a subgroup meta-analysis. (3) Results: The systematic review identified 27 studies eligible for the quantitative synthesis. Partially vaccinated individuals presented a RRR = 73% (95%CI = 59–83%) for positive SARS-CoV-2 PCR (RR = 0.27) and a RRR=79% (95%CI = 30–93%) for symptomatic SARS-CoV-2 PCR (RR = 0.21). Fully vaccinated individuals showed a RRR = 94% (95%CI = 88–98%) for SARS-CoV-2 positive PCR (RR = 0.06) compared to unvaccinated individuals. The full BNT162b2 vaccination protocol achieved a RRR = 84–94% against any SARS-CoV-2-positive PCR and a RRR = 68–84% against symptomatic positive PCR. (4) Conclusions: The meta-analysis results suggest that full vaccination might block transmission. In particular, the risk of SARS-CoV-2 infection appeared higher for non-B.1.1.7 variants and individuals aged ≥69 years. Considering the high level of heterogeneity, these findings must be taken with caution. Further research on SARS-CoV-2 vaccine effectiveness against emerging SARS-CoV-2 variants is encouraged.

Since December 2020, infections with SARS-CoV-2 and the associated disease, COVID-19 (coronavirus disease 2019), have spread worldwide. On 11 March 2020, the WHO declared the COVID-19 outbreak a pandemic. The coronavirus disease 2019 (COVID- 19) has represented a serious threat to public health, with nearly 4,038,342 deaths and 186,821,815 confirmed cases reported globally. As of 5 July 2021, 3,114,766,865 vaccine doses had been administered [1] .

Coronaviruses (CoVs), including SARS-CoV, MERS-CoV, and SARS-CoV-2, are positivesense, single-stranded RNA viruses with four structural proteins (S protein, envelope (E) protein, membrane (M) protein, and nucleocapsid (N) protein) [2] . The spike protein of SARS-CoV-2 binds to ACE2 receptors on target cells and acts as an immunodominant antigen, eliciting both antibody and T-cell responses [3] .

To date, the most important COVID-19 candidate vaccines have been developed by using mRNA technology, adenoviral vectors, inactivated virus, and adjuvants [4] .

The systematic review started on March 2021 and concluded on 15 May 2021. The main purpose of this research was to assess the risk of COVID-19 occurrence (any positive RT-PCR test) among vaccinated subjects. Moreover, we aimed at quantifying the risk of developing symptomatic COVID-19 after vaccination. The evaluation was narrowed to the BNT162b2, mRNA-1273, ChAdOx1/AZD1222, and Ad26.COV2.S vaccines. The study followed the PRISMA 2020 statement [9] . The systematic review was prospectively registered on PROSPERO (n. CRD42021240143).

The search strategy is presented in detail in Supplementary Material file 1. We used seven different web engines, including early-stage research platforms: PubMed, Cochrane, clinicaltrial.gov, COVID-NMA, medRxiv, SSRN, and Authorea. No restriction on language, setting or publication date was imposed.

The papers were selected initially according to their titles and abstracts. The full texts that were suitable for the quantitative synthesis were collected in an Excel database for the data extraction. The exclusion criteria included lack of suitable data and study design. We narrowed the quantitative synthesis to non-experimental studies only.

The data extraction was performed by two authors, independently. The results of the respective phase III RCTs were used as references for data extraction [6, 7, 10, 11] . The information derived from each full text was classified into four categories: (1) outcome, (2) study characteristics (design, publication status, year of publication), (3) participants characteristics (mean age, severity of COVID-19 symptoms, dose of SARS-CoV-2 vaccine, SARS-CoV-2 lineage) and (4) risk of bias. Disagreements were resolved by discussion or, if necessary, consultation with a third party.

The primary endpoint of this research was to measure the overall relative risk (RR) of any SARS-CoV-2 infection 14 days after the first dose (partially vaccinated), seven days after the second dose (fully vaccinated), and 14 days after at least one dose uptake.

As a secondary endpoint, we aimed to measure the risk of symptomatic infection following SARS-CoV-2 vaccination, and, if possible, to quantify the risk of hospitalization and death after partial and full vaccination protocol.

To this end, any COVID-19 infection, either symptomatic or asymptomatic, confirmed through PCR was considered as a COVID-19 case. Studies that did not feature the routine performance a PCR test after vaccination were excluded. Documented SARS-CoV-2 infections (any positive rtPCR) at the baseline were excluded from the effectiveness analysis.

The risk of infection between vaccinated and unvaccinated groups was calculated as relative risk (RR). In order to quantify how much SARS-CoV-2 vaccination reduced the risk of infection relative to the control group, we computed the relative risk reduction (RRR) according to the formula 100*(1−RR)

The primary analysis included the RR of any new positive infection episode among vaccinated and unvaccinated groups confirmed through PCR test. The impact of SARS-CoV-2 vaccination on infection severity was investigated in secondary analysis as the RR of symptomatic positive PCR and, possibly, the RR of hospitalization and death following partial and full vaccination protocols. Both self-reported symptoms and symptoms ascertained through clinical visit were included.

Finally, COVID-19 episodes were classified as compatible with the B.1.1.7 variant and incompatible (i.e., B.135, non-B.1.1.7, unspecified).

All adults eligible to undergo SARS-CoV-2 mass vaccination were included in the experimental group. For the definition of cases in the vaccinated groups, we referred to Polack et al., Phase III RCT [6] . The same rules were applied to ChAdOx1/AZD1222 and mRNA-1273. Concerning ChAdOx1/AZD1222, only a small number of studies on VE after the first dose administration were available, as the vaccine was approved later for the emergency rollout [10] . The single-dose administration of Ad26.COV2.S was considered as full vaccination protocol, although the induction time lasts two weeks ( Figure 1 ) [11] . The effectiveness of the vaccines against positive SARS-COV-2 with at least one dose was tested 14 days after the administration of the first dose. 

In the general population, the mean incubation period for COVID-19 symptom onset was estimated as 5.8 days [13] . So far, vaccination has been prioritized towards older and at-risk individuals; this aspect might have biased the comparison with the unvaccinated The 14 days following the administration of the first dose were designated as induction time because they were considered as the minimum time spell needed to develop COVID-19 immunity. Therefore, regardless of vaccine technology, SARS-CoV-2 cases occurring within two weeks of the first dose were not considered as cases within the vaccinated population [6, 12] . In order to assess the effectiveness of the full vaccination protocol, the upper follow-up limit was set at seven days after the second dose. Hence, only SARS-CoV-2 cases occurring at least one week after the second dose were attributed to a lack of vaccine protection [6] . Moreover, cases occurring within these seven days were still attributed to the first dose's effectiveness (Figure 1 ).

The effectiveness of the vaccines against positive SARS-COV-2 with at least one dose was tested 14 days after the administration of the first dose.

In the general population, the mean incubation period for COVID-19 symptom onset was estimated as 5.8 days [13] . So far, vaccination has been prioritized towards older and at-risk individuals; this aspect might have biased the comparison with the unvaccinated cohort, which generally was younger and healthier. Overall, cohort studies drew the control group from the same population as the vaccinated group, while in a few cases the cohort was the same community observed before and after the vaccination. For reasons mostly related to a dearth of detailed information on unvaccinated individuals, the beginning of the follow-up for the control group was established at T0 ( Figure 1 ).

In order to assess the risk of bias, we employed the Newcastle-Ottawa Scale (NOS) in cohort and case-control studies, and the NOS adapted for cross-sectional studies [14] .

We performed a frequentist meta-analysis using the inverse variance (IV) method [15] . In addition, the Mantel-Haenszel (MH) method was applied because we expected to encounter sparse data, especially in estimating the RR of symptomatic cases after vaccination. The analysis was executed on R (version 4.0.5).

The random effect (RE) meta-analysis was considered more appropriate to estimate the overall effect size. However, both fixed effects (FE) and RE outputs were reported. In order to test for the overall heterogeneity, both the Cochran's Q (chi-squared statistic) and I-squared values (I 2 ) were calculated. The Der Simonian and Laird method estimated the between-study variance τ 2 [16] .

As between-study heterogeneity can be caused by studies with extreme effect sizes, low-quality or small sample sizes, the following analyses were performed: outliers test, influence analyses [17] , Baujat Plot analysis [18] , leave-one-out analysis, and Graphic Display of Heterogeneity (GOSH) with Knapp-Hartung adjustment [19] (Supplementary Material file 2).

The subgroup meta-analysis, which enclosed vaccine type, quality, age of vaccinated population, and SARS-CoV-2 lineage detected through PCR tests, was performed to identify further heterogeneity sources [19] . As we expected a limited number of studies in some subgroups (n < 10), we opted for a FE model in the subgroup meta-analysis. The subgroup levels were considered fixed and exhaustive. Nevertheless, in order to capture discrepancies that might yield inconclusive results, more conservative methods, such as RE and mixed model, were added to the output (reported in Supplementary Material file 3.1).

In order to test whether publication biases influenced the study conclusions, we performed a funnel plot (Supplementary Material file 3.2) [20] .

The web search provided 7760 unduplicated records. Overall, 31 studies were selected for data extraction and qualitative data synthesis (Supplementary Material file 4, Figure S22 ). There were 24 cohort studies, two case-control, one cross-sectional, one matched observational, and three test negative case-control. Ten studies were performed in the UK, eight in the US, and five in Israel. The RR of SARS-CoV-2 infection among health care workers (HCWs) was examined by thirteen studies; four studies analyzed long-term care facility (LCTF) residents and nine the general population ( Supplementary Material file 4, Table S8 ).

In the following meta-analysis, an RE model with inverse variance (IV) method was used throughout. The evidence synthesis was based on RR as the measure of effect. Additionally, the FE estimates and the Mantel-Haenszel outputs are reported in Table 1 . The between-study variance τ 2 was computed for each RE meta-analysis. Overall, 27 out of 31 studies were included in the quantitative analyses. Four studies met the inclusion criteria for the systematic review but were not included in the metaanalyses [41, [44] [45] [46] .

Despite some small sample sizes and the small number of infection cases expected among vaccinated individuals, no study presented zero cells during the data extraction. The ChAdOx1/AZD1222 effectiveness was investigated in 4 out of 27 studies [22, 23, 42, 43] , while only one study estimated Ad26.COV2.S effectiveness [38] .

The Newcastle-Ottawa Quality Assessment Scale showed a satisfactory quality score, with a median of six, a minimum of five, and a maximum of eight ( Supplementary Material file 4, Table S9 ).

The meta-analysis on first-dose VE against any positive PCR pooled 17 studies and 22 entries. Jones' analysis spanned a follow-up period of two weeks (A) and, additionally, an extended period of six weeks (B) [21] . Lopez-Bernal et al. evaluated the effectiveness of one and two doses of the BNT162b2 vaccine on adults aged ≥80 years (A80 and B80) and ≥70 years (B70 and C70) in England. Additionally, the effectiveness of a single dose of ChAdOx1/AZD1222 (A70) and at least a single dose of BNT162b2 (C70) were tested on adults aged ≥70 years (A70) [22] . Shotri et al. estimated the protective effects of the first dose of BNT162b2 (A) and ChAdOx1/AZD1222 (B) against any SARS-CoV-2 infection [23] . One study from Denmark investigated BNT162b2 VE in two cohorts, namely long-term care facility residents (R) and HCWs (H) [25] . Abu Raddad et al. measured the effectiveness of BNT162b2 against B.1.1.7 (A) and B.1.351 (B), which became predominant within Qatar in early 2021, and any different (C) SARS-CoV-2 variants [25] . According to the RE meta-analysis results (IV), partially vaccinated individuals showed an RRR of 73% for any SARS-CoV-2 positive PCR compared to unvaccinated individuals (RR = 0.27). The FE model yielded lower effectiveness estimates after partial vaccination (RR = 0.41; RRR = 59%). Although both FE and RE (IV) showed a statistically significant result (p < 0.0001), the heterogeneity was considerable (τ 2 = 1.08; H = 18.43 (17.52; 19.38 ; I 2 = 99.7%); therefore, the true size of the effect remained uncertain (Table 1, Figure 2a ).

In the evaluation of the RR for any positive SARS-CoV-2 PCR following the full vaccination protocol, only data on mRNA vaccines (BNT162b2 and mRNA-1273) were available. Sixteen studies and seventeen entries were included in this meta-analysis. One study evaluated Ad26.COV2.S effectiveness, while Pritchard indistinctly assessed the effectiveness of ChAdOx1/AZD1222 and BNT162b2 after the full vaccination protocol [26] . In Abu Raddad's study, the data on PCR tests were available after full vaccination protocol with BNT162b2 (two weeks after second dose administration) [25] . Four studies did not distinguish between the BNT162b2 and mRNA-1273 vaccines [27] [28] [29] [30] .

Based on the RE meta-analysis results (IV), the RRR for any positive SARS-CoV-2 PCR following the full vaccination protocol was 94% (95%CI = 88-98%) with RR = 0.06, compared to unvaccinated individuals (Table 1, Figure 2b ). FE yielded a RRR = 88%. Although both FE and RE showed a statistically significant protective effect of full COVID-19 vaccination (p < 0.0001), we should be cautious about drawing conclusions as to the true size effect because the heterogeneity was considerable (τ 2 = 2.6; H = 10.07 (9.22; 11.00); I 2 = 99.0%). The test for heterogeneity was significant at 1% (Q = 1621.96, p = 0). (Table 1, Figure 2b) .

A longer follow-up period was available in the VE estimation after at least one dose. For this group, the mean length of follow-up was 54 days after the administration of the first dose. The meta-analysis on the RR of testing positive for any SARS-CoV-2 PCR included 18 entries and 14 studies. Menni et al. analyzed SARS-CoV-2 infection in individuals who received one or two doses of BNT162b2 (A) and ChAdOx1/AZD1222 (B) [43] . The results of the RE meta-analysis (IV) produced a significant RRR, 84%, for vaccinated individuals compared to unvaccinated individuals, with RR = 0.26. The test for heterogeneity was significant at 1% (Q = 1030.89, p < 0.0001) (Table 1, Figure 2c ). 

In order to assess the effectiveness of SARS-CoV-2 vaccination against symptomatic COVID-19 infection, we performed two meta-analyses, in partially and fully vaccinated individuals (Table 1) . The meta-analysis on the full vaccination protocol included only studies estimating VE among individuals aged <69 years. The meta-analyses on VE against symptomatic SARS-CoV-2 infection pooled 9 studies and 17 entries in total (9 and 8 entries pertained to partial and full vaccination protocols, respectively). The IV method 

In order to assess the effectiveness of SARS-CoV-2 vaccination against symptomatic COVID-19 infection, we performed two meta-analyses, in partially and fully vaccinated individuals (Table 1) . The meta-analysis on the full vaccination protocol included only studies estimating VE among individuals aged <69 years. The meta-analyses on VE against symptomatic SARS-CoV-2 infection pooled 9 studies and 17 entries in total (9 and 8 entries pertained to partial and full vaccination protocols, respectively). The IV method displayed significant results for both FE and RE (p < 0.05). According to the RE results (IV), partial vaccination achieved an RRR = 78% (95% CI = 31-93%) for symptomatic SARS-CoV-2 infection compared to unvaccinated, while fully vaccinated subjects exhibited a RRR = 94% (95% CI = 84-98%) (Figure 3a,b) . FE estimates of RRR were lower for both partially and fully vaccinated individuals (RRR = 51% and RRR = 76%, respectively). Both FE and RE were statistically significant (p < 0.05). 

Concerning hospitalization risk, an RE meta-analysis over two studies [22, 31] produced an RR = 0.38 (95%CI = 0.2719; 0.5242) after a full BNT162b2 vaccination protocol. (Q = 2.04, p = 0.1529; I2 = 51.1% (0.0%; 87.6%)). Based on Bernal et al.'s data, at least one dose of the BNT162b2 vaccine ensured an RRR of death equal to 83%, compared to unvaccinated individuals [23] .

Considering the substantial heterogeneity and the significance of the heterogeneity test, we could not be overly confident that the RR estimate would be robust in every context; therefore, we performed a sensitivity analysis to address the between-study heterogeneity (Supplementary Material file 2).

The L' Abbé plots in Figure 4 confirmed that, overall, infection events were in favor 

Concerning hospitalization risk, an RE meta-analysis over two studies [22, 31] produced an RR = 0.38 (95%CI = 0.2719; 0.5242) after a full BNT162b2 vaccination protocol. (Q = 2.04, p = 0.1529; I 2 = 51.1% (0.0%; 87.6%)). Based on Bernal et al.'s data, at least one dose of the BNT162b2 vaccine ensured an RRR of death equal to 83%, compared to unvaccinated individuals [23] .

Considering the substantial heterogeneity and the significance of the heterogeneity test, we could not be overly confident that the RR estimate would be robust in every context; therefore, we performed a sensitivity analysis to address the between-study heterogeneity (Supplementary Material file 2).

The L' Abbé plots in Figure 4 confirmed that, overall, infection events were in favor of the control group (unvaccinated). Concerning symptomatic SARS-CoV-2 infections, the L'Abbè plots in Figure 4d ,e confirmed that the infection rates were greater in the unvaccinated groups than in the partially or fully vaccinated groups. . The further the circle from the line of no effect, the greater the difference of event rates between intervention and control arms.

The GOSH analysis did not indicate a clear cluster that contributed to the pooled imbalance (Supplementary Material file 2). In the subgroup meta-analysis, we grouped the studies by type of vaccine, mean age of sample (≥69 and <69 years), SARS-CoV-2 lineage, and bias assessment performed through Newcastle-Ottawa Scale (NOS). For simplicity, the NOS quality score was classified as satisfactory (NOS ≤ 6) and good (NOS > 6).

The FE subgroup meta-analysis partly explained the between-study heterogeneity because all the subgroups produced a significant Q test (p < 0.0001). However, considering the residual heterogeneity and the small number of studies in several subgroups, the results must be interpreted with caution.

The first subgroup analysis compared different vaccine technologies. Based on the FE subgroup estimates, the RRR after the first dose of BNT162b2 ranged from 46% to 49% for any positive SARS-COV-2 PCR and from 35% to 32% for symptomatic events. This discrepancy might be explained by the low number of studies included in the subgroup evaluating VE against symptomatic SARS-COV-2 infection (n = 6) with respect to the group evaluating the effectiveness of the BNT162b2 vaccine against any positive PCR (n = 16) ( Table 2 ). In FE, the RR estimates were greater in higher-quality studies (NOS > 6), whether symptomatic or not (RR = 0.12 and RR = 0.07 respectively) compared to lower- Fully vaccinated, symptomatic positive SARS-CoV-2 PCR RR ≥ 7 days from full vaccination (e). The sizes of the plotted circles are proportional to the precision of the studies. The dashed lines mark the overall estimate of the log risk for RE (light) and FE (bold). The further the circle from the line of no effect, the greater the difference of event rates between intervention and control arms.

The GOSH analysis did not indicate a clear cluster that contributed to the pooled imbalance (Supplementary Material file 2). In the subgroup meta-analysis, we grouped the studies by type of vaccine, mean age of sample (≥69 and <69 years), SARS-CoV-2 lineage, and bias assessment performed through Newcastle-Ottawa Scale (NOS). For simplicity, the NOS quality score was classified as satisfactory (NOS ≤ 6) and good (NOS > 6).

The FE subgroup meta-analysis partly explained the between-study heterogeneity because all the subgroups produced a significant Q test (p < 0.0001). However, considering the residual heterogeneity and the small number of studies in several subgroups, the results must be interpreted with caution.

The first subgroup analysis compared different vaccine technologies. Based on the FE subgroup estimates, the RRR after the first dose of BNT162b2 ranged from 46% to 49% for any positive SARS-COV-2 PCR and from 35% to 32% for symptomatic events. This discrepancy might be explained by the low number of studies included in the subgroup evaluating VE against symptomatic SARS-COV-2 infection (n = 6) with respect to the group evaluating the effectiveness of the BNT162b2 vaccine against any positive PCR (n = 16) ( Table 2 ). In FE, the RR estimates were greater in higher-quality studies (NOS > 6), whether symptomatic or not (RR = 0.12 and RR = 0.07 respectively) compared to lower-quality studies (NOS ≤ 6, RR = 0.67 and RR = 0.48, respectively). Partial vaccination effectiveness appeared lower within studies that examined older populations (≥69 years) compared to younger, either on symptomatic PCR (RR = 0.40 and RR = 0.49, respectively) or any positive PCR (RR = 0.29 and RR = 0.48, respectively).

Regarding the FE subgroup meta-analysis in fully vaccinated individuals (Table 2) , BNT162b2 produced an RRR = 83% against any positive PCR and an RRR = 68% against symptomatic SARS-CoV-2 infections. The RE and the mixed model produced an RRR of 94% against any positive PCR and an RRR of 84% against symptomatic COVID-19. Further, the FE subgroup meta-analysis relative to the bias assessment (NOS) showed a greater RR for lower-quality studies compared to higher-quality studies (RR = 0.01 in symptomatic PCR and an RR of 0.33 in any positive PCR meta-analyses, respectively). Full vaccination effectiveness against any positive PCR appeared slightly lower in studies that examined older populations (≥ 69 years) compared to younger populations (RR = 0.15 and RR = 0.12, respectively).

Concerning partial vaccination effectiveness against SARS-CoV-2 variants, the RRR of any positive PCR for B.1.1.7 was 60% in FE and 81% in the mixed model. The RRR increased to 85% in FE and 92% in the mixed model after full vaccination. Although only 'Abu Raddad' assessed the effectiveness of BNT162b2 against the B.1.351 lineage [25] , the RRR was lower for the B.1.351 variant in both partially (RRR = 9%) and fully vaccinated subjects (RRR = 60%). Regarding symptomatic PCR, the RRR in fully vaccinated individuals remained greater for B.1.1.7 (RRR = 86%) compared to B.1.351 (RRR = 64%) ( Table 2 , a, b, e).

The funnel plots are displayed in Supplementary Material file 3.2 ( Figure S21 ). The contour-enhanced funnel plots do not exhibit publication bias. The Egger test indicated the presence of funnel plot asymmetry in the meta-analysis of symptomatic SARS-CoV-2 RR after the full vaccination protocol (intercept = −11.271, p = 0.026), although it may lack the statistical power to detect bias because the number of studies was small (n <10). 

Despite the considerable between-study heterogeneity, our findings provided evidence that any COVID-19 vaccine is highly effective outside the controlled conditions of clinical trials. Overall, SARS-CoV-2 vaccination is effective at reducing the number of new COVID-19 cases, either asymptomatic or symptomatic, with the greatest benefit achieved after completing the full vaccination protocol. Fewer real-world data were available for ChAdOx1/AZD1222 due to its later approval for deployment. However, vaccination with ChAdOx1/AZD1222 resulted in the rare onset of immune thrombotic thrombocytopenia; therefore, it underwent an additional careful monitoring process by European Union (EU) regulatory authorities [52] .

According to our findings, partially vaccinated individuals were only a quarter as likely (RR = 0.26) to develop any SARS-CoV-2 infection as unvaccinated individuals, while their overall risk of symptomatic infection was slightly lower (RR = 0.22). The full vaccination schedule was 94% effective against asymptomatic and symptomatic positive PCR tests. Concerning hospitalization risk, a meta-analysis over two studies [22, 31] yielded an RRR of 62% after the full BNT162b2 vaccination protocol. Concerning mortality risk, our findings corroborated Bernal et al.'s study [22] , conducted over 7.5 million adults aged 70 years and older in the UK. At least one dose of BNT162b2 was approximately 83%, effective at preventing death compared to no vaccination, while there was insufficient follow-up to assess ChAdOx1/AZD1222's impact on mortality because of its delayed rollout.

The subgroup analysis did not find any significant difference in effectiveness between the mRNA and the modified adenovirus vaccines, in part because the evidence, especially for ChAdOx1/AZD1222, was still very scarce. Up to 15 May, only two studies evaluated ChAdOx1/AZD1222 [22, 23, 41, 42] , and only one Ad26.COV2.S effectiveness [38] . As expected, COVID-19 VE appeared higher in adults aged <69 years. The evaluation of VE against the SARS-CoV-2 variants remained a key point that lacked robust results. In fact, only one study investigated the effectiveness of BNT162b2 against two variants of concern [25] . The RR of infection following the full vaccination protocol appeared larger for the B.1.351 (South Africa) variant compared to B.1.1.7 (UK). Although the evidence suggested a lower VE against SARS-COV-2 variants of concern, our findings were not sufficient to assert that full COVID-19 vaccination protocol is 60% effective against B.1.351 and 94% effective against B.1.1.7.

Although imprecise, the real-world data confirmed the experimental evidence and suggested that the vaccine offers mild protection against emerging SARS-CoV-2 variants as well as within the elderly population. By including non-randomized studies, our research aimed to address questions not answered by clinical trials, such as VE within different population subgroups. Moreover, the effect of waning immunity on effectiveness needs long-term investigation, which appears more compatible with a non-RCT study design.

The present study features several limitations. The subgroup analysis and the betweenstudy heterogeneity analysis were not able to reduce the overall heterogeneity. The sparsity of data at patient level did not allow further investigation of unobserved sources through a metaregression. Undeniably, the heterogeneity may have stemmed from the observational design of the included studies and, unless through a randomized process, it is unlikely to be reduced in any circumstances.

Five studies did not test their vaccinated cohorts systematically during follow-up. The absence of active laboratory surveillance of vaccinated individuals might have resulted in an underestimation of asymptomatic cases. However, we did not consider a different rate of testing as a bias of concern in terms of RR estimate because both vaccinated and unvaccinated individuals underwent the same case investigation and contact tracing protocol. In addition, when systematic testing was not performed, asymptomatic testing was available to different extents: workplace exposures (HCWs), out-of-state travelers or per-request [28] .

Ultimately, asymptomatic cases not confirmed by PCR test, as well as false negatives, might represent a source of bias in both vaccinated and unvaccinated groups [28, 34] . Finally, public health mitigation measures might contribute to the underestimation of SARS-CoV-2 vaccine effectiveness. Unfortunately, the extent to which primary prevention restrictions affected VE was beyond the scope of this study.

By way of conclusion, we can state that a significant reduction in the RR of asymptomatic infection within partially vaccinated individuals was not corroborated by sufficient statistical robustness for the results to be generalizable. However, full vaccination effectiveness against symptomatic and asymptomatic SARS-CoV-2 infection risk confirmed the RCT results, leading to the same RR estimates but larger CI. In order to investigate additional sources of heterogeneity that might affect the validity of the meta-analysis results, further research on real-world SARS-CoV-2 vaccine effectiveness is encouraged. Ultimately, our findings support the maximization of full vaccination coverage. Additional evidence about the impact of SARS-CoV-2 variants on vaccine effectiveness is vital in order to monitor mutations associated with vaccine escape. Table S9 : Quality assessment Newcastle-Ottawa Scale (NOS). Figure S21 : Funnel plot and Eggers' test. Figure 

The authors declare no conflict of interest.

Emerging coronaviruses: Genome structure, replication, and pathogenesis

Broad and strong memory CD4 + and CD8 + T cells induced by SARS-CoV-2 in UK convalescent individuals following COVID-19

Safety and immunogenicity of ChAdOx1 nCoV-19 vaccine administered in a prime-boost regimen in young and old adults (COV002): A single-blind, randomised, controlled, phase 2/3 trial

Safety and Efficacy of the BNT162b2 mRNA COVID-19 Vaccine

Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine

The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration

Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: An interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK

Safety and Efficacy of Single-Dose Ad26.COV2.S Vaccine against COVID-19

Incubation period of COVID-19: A rapid systematic review and meta-analysis of observational research

The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomised Studies in Meta-Analyses

Controlling the risk of spurious findings from meta-regression

Outlier and influence diagnostics for meta-analysis

A graphical method for exploring heterogeneity in meta-analyses: Application to a meta-analysis of 65 trials

GOSH-A graphical display of study heterogeneity

Bias in meta-analysis detected by a simple, graphical test

Single-dose BNT162b2 vaccine protects against asymptomatic SARS-CoV-2 infection

Effectiveness of the Pfizer-BioNTech and Oxford-AstraZeneca vaccines on COVID-19 related symptoms, hospital admissions, and mortality in older adults in England: Test negative case-control study

Vaccine effectiveness of the first dose of ChAdOx1 nCoV-19 and BNT162b2 against SARS-CoV-2 infection in residents of long-term care facilities in England (VIVALDI): A prospective cohort study

Vaccine effectiveness after 1st and 2nd dose of the BNT162b2 mRNA COVID-19 Vaccine in long-term care facility residents and healthcare workers-A Danish cohort study

National Study Group for COVID-19 Vaccination. Effectiveness of the BNT162b2 Covid-19 Vaccine against the B.1.1.7 and B.1.351 Variants

Impact of vaccination on new SARS-CoV-2 infections in the United Kingdom

FDA-authorized mRNA COVID-19 vaccines are effective per real-world evidence synthesized across a multi-state health system

COVID-19 vaccine impact on rates of SARS-CoV-2 cases and post vaccination strain sequences among healthcare workers at an urban academic medical center: A prospective cohort study

Effectiveness of Pfizer-BioNTech and Moderna Vaccines Against COVID-19 Among Hospitalized Adults Aged ≥65 Years-United States

Effectiveness of Messenger RNA Coronavirus Disease 2019 (COVID-19) Vaccines Against Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Infection in a Cohort of Healthcare Personnel

Effects of BNT162b2 mRNA vaccine on COVID-19 infection and hospitalisation amongst older people: Matched case control study for England

Effectiveness of the first dose of BNT162b2 vaccine to preventing COVID-19 in healthcare personnel

COVID-19 vaccine coverage in health-care workers in England and effectiveness of BNT162b2 mRNA vaccine against infection (SIREN): A prospective, multicentre, cohort study

Effectiveness of the Pfizer-BioNTech COVID-19 Vaccine Among Residents of Two Skilled Nursing Facilities Experiencing COVID-19 Outbreaks-Connecticut

COVID-19 Registries Study Group2. Direct and Indirect Effectiveness of mRNA Vaccination against Severe Acute Respiratory Syndrome Coronavirus 2 in Long-Term Care Facilities

Effectiveness of BNT162b2 and ChAdOx1 nCoV-19 COVID-19 vaccination at preventing hospitalisations in people aged at least 80 years: A test-negative, case-control study

Association Between Vaccination With BNT162b2 and Incidence of Symptomatic and Asymptomatic SARS-CoV-2 Infections Among Health Care Workers

Analysis of the Effectiveness of the Ad26.COV2.S Adenoviral Vector Vaccine for Preventing COVID-19

Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: What are they and which is better?

Impact and effectiveness of mRNA BNT162b2 vaccine against SARS-CoV-2 infections and COVID-19 cases, hospitalisations, and deaths following a nationwide vaccination campaign in Israel: An observational study using national surveillance data

An observational cohort study on the incidence of SARS-CoV-2 infection and B.1.1.7 variant infection in healthcare workers by antibody and vaccination status

Vaccine side-effects and SARS-CoV-2 infection after vaccination in users of the COVID Symptom Study app in the UK: A prospective observational study

Symptom study app provides real-world data on COVID-19 vaccines

Interim Estimates of Vaccine Effectiveness of BNT162b2 and mRNA-1273 COVID-19 Vaccines in Preventing SARS-CoV-2 Infection Among Health Care Personnel, First Responders, and Other Essential and Frontline Workers-Eight

Interim findings from first-dose mass COVID-19 vaccination roll-out and COVID-19 hospital admissions in Scotland: A national prospective cohort study

Early rate reductions of SARS-CoV-2 infection and COVID-19 in BNT162b2 vaccine recipients

BNT162b2 mRNA Covid-19 Vaccine Effectiveness among Health Care Workers

BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Mass Vaccination Setting

Early Evidence of the Effect of SARS-CoV-2 Vaccine at One Medical Center

BioNTech/Pfizer) vaccine in preventing SARS-CoV-2 infection among healthcare workers, Treviso province

Impact of the COVID-19 Vaccine on Asymptomatic Infection Among Patients Undergoing Pre-Procedural COVID-19 Molecular Screening

Vaccine: EMA Finds Possible Link to very Rare Cases of Unusual Blood Clots with Low Blood Platelets