key: cord-0834602-qeqmx59f
authors: Kow, Chia Siang; Hasan, Syed Shahzad
title: Real-world effectiveness of BNT162b2 mRNA vaccine: a meta-analysis of large observational studies
date: 2021-07-09
journal: Inflammopharmacology
DOI: 10.1007/s10787-021-00839-2
sha: e6f4316f947d22d730d8d2ecc27ae8f302d8ad61
doc_id: 834602
cord_uid: qeqmx59f

This paper aims to summarize through meta-analyses the overall vaccine effectiveness of the BNT162b2 mRNA vaccine from observational studies. A systematic literature search with no language restriction was performed in electronic databases to identify eligible observational studies which reported the adjusted effectiveness of the BNT162b2 mRNA vaccine to prevent RT-PCR confirmed COVID-19. Meta-analyses with the random-effects model were used to calculate the pooled hazard ratio (HR) and pooled incidence rate ratio (IRR) at 95% confidence intervals, and the vaccine effectiveness was indicated as (pooled HR − 1)/HR or (pooled IRR − 1)/IRR. Nineteen studies were included for this meta-analysis. The meta-analysis revealed significant protective effect against RT-PCR confirmed COVID-19 ≥ 14 days after the first dose, with vaccine effectiveness of 53% (95% confidence interval 32–68%), and ≥ 7 days after the second dose, with vaccine effectiveness of 95% (95% confidence interval: 96–97%). Despite its effectiveness, reporting vaccine safety data by relevant stakeholders should be encouraged as BNT162b2 mRNA is a new vaccine that has not gained full approval. There have been limited data about vaccine effectiveness among immunocompromised patients; thus, the vaccine should be used cautiously in this patient population.

The global rollout of vaccines against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) offers a glimmer of hope toward ending the coronavirus disease 2019 (COVID-19) pandemic. As of the time of writing, there have been with more than 1.7 billion people worldwide received at least one dose of any COVID-19 vaccine, and over 790 million people worldwide are fully vaccinated (Our World in Data 2021).

The phase 3 randomized controlled trial of the BNT162b2 mRNA vaccine against SARS-CoV-2 demonstrated the efficacy of 95% in preventing symptomatic COVID-19, which has led to the emergency conditional approval of the vaccine in many countries (Polack et al. 2020 ). However, it should be noted that the clinical trial was performed in a highly controlled setting that may not simulate the realworld mass rollout of COVID-19 vaccines.

Therefore, it is imperative to determine the populationlevel vaccine effectiveness from the mass vaccination campaigns and to report data on the safety aspects of vaccines. This paper aims to summarize through meta-analyses the overall effectiveness of the BNT162b2 mRNA vaccine from large observational studies, which could be important to inform the development of the public health policy related to mass vaccination.

A systematic literature search with no language restriction was performed in electronic databases, including PubMed, Google Scholar, Scopus, and preprint servers (medRxiv, Research Square, SSRN) , to identify eligible studies published up to June 05, 2021. The search strategy was built based on the Inflammopharmacology * Chia Siang Kow chiasiang_93@hotmail.com 1 following keywords and MeSH terms: "BNT162b2", "Pfizer", "BioNTech", "mRNA vaccine", "mRNA vaccination", and "effectiveness". The reference lists of relevant articles were also reviewed to retrieve additional studies. Two investigators (CSK and SSH) independently performed the literature screening to identify eligible studies. Studies eligible for inclusion were observational studies of any design (case-control, case-cohort, and prospective cohort), which reported the effectiveness of the BNT162b2 mRNA vaccine to prevent reverse transcription-polymerase chain reaction (RT-PCR) confirmed COVID-19 (through comparison between vaccinated and unvaccinated individuals) and adjusted for covariates. For two or more studies which utilized the same data source for their investigations on vaccine effectiveness, we included the one that performed analysis on the latest record. We excluded randomized trials, studies that reported unadjusted effectiveness estimates, studies that reported only non-specific outcomes such as COVID-19-related mortality or COVID-19-related hospitalization, studies where RT-PCR did not use to confirm the diagnosis of COVID-19, and studies that reported vaccine effectiveness against a specific variant(s) of SARS-CoV-2.

Our outcome of interest, namely vaccine effectiveness, is defined as a relative reduction in RT-PCR risk confirmed COVID-19 in vaccinated individuals compared with unvaccinated individuals (Weinberg and Szilagyi 2010) . Each included study was independently evaluated by two investigators (CSK and SSH), who also extracted the study characteristics. Study characteristics extracted had the first author's surname, study design, country, sample population, number of participants, the incidence of COVID-19 in both vaccinated and unvaccinated individuals, and adjusted vaccine effectiveness estimates and covariates adjusted. Two investigators (CSK and SSH) assessed the quality of included observational studies using the Newcastle-Ottawa Scale, with a score of > 7 indicating high quality (Wells et al. 2013) .

Meta-analyses with the random-effects model were used to calculate the pooled hazard ratio (HR), pooled incidence rate ratio (IRR), or pooled odds ratio (OR) at 95% confidence intervals, comparing the incidence of RT-PCR confirmed COVID-19 in vaccinated participants relative to unvaccinated participants, when there were three or more studies reporting the same type of effect measure (either HR, IRR, or OR). The vaccine effectiveness was indicated as (pooled HR − 1)/HR, (pooled IRR − 1)/IRR or (pooled OR − 1)/OR, together with a 95% confidence interval. We examined the heterogeneity between studies using the I 2 statistics and the χ 2 test, with significant heterogeneity set at > 50% and P < 0.10. All analyses were performed using Meta XL, version 5.3 (EpiGear International, Queensland, Australia).

Our literature search yielded 712 abstracts. After deduplication and application of the eligibility criteria, 38 relevant articles were shortlisted for inclusion through full-text examination ( Fig. 1 ). Of these, 19 studies were excluded since they either did not report vaccine effectiveness, reported non-specific outcomes such as COVID-19-related mortality and COVID-19-related hospitalization, or reported unadjusted effectiveness estimates. Therefore, 19 studies (Angel et al. 2021; Björk et al. 2021; Cabezas et al. 2021; Chung et al. 2021; Dagan et al. 2021; Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Gras-Valentí et al. 2021; Haas et al. 2021; Hall et al. 2021; Lopez Bernal et al. 2021; Mason et al. 2021; Monge et al. 2021; Pritchard et al. 2021; Regev-Yochay et al. 2021; Shrotri et al. 2021; Swift et al. 2021; Thompson et al. 2021) were included for this meta-analysis; 12 studies (Chung et al. 2021; Dagan et al. 2021; Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Gras-Valentí et al. 2021; Haas et al. 2021; Lopez Bernal et al. 2021; Mason et al. 2021; Monge et al. 2021; Regev-Yochay et al. 2021) were retrospective in design with seven database reviews (Dagan et al. 2021; Emborg et al. 2021; Glampson et al. 2021; Haas et al. 2021; Mason et al. 2021; Monge et al. 2021; Swift et al. 2021) , three retrospective case-control studies (Chung et al. 2021; Gras-Valentí et al. 2021; Lopez Bernal et al. 2021) , and two retrospective cohort studies (Fabiani et al. 2021; Regev-Yochay et al. 2021) ; the remaining seven studies (Björk et al. 2021; Cabezas et al. 2021; Hall et al. 2021; Menni et al. 2021; Shrotri et al. 2021; Thompson et al. 2021; Pritchard et al. 2021) were prospective cohort studies (n = 6) (Cabezas et al. 2021; Hall et al. 2021; Menni et al. 2021; Shrotri et al. 2021; Thompson et al. 2021; Pritchard et al. 2021 ) and prospective database review (n = 1) (Björk et al. 2021) . The included studies (Björk et al. 2021; Dagan et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Haas et al. 2021; Hall et al. 2021; Mason et al. 2021; Menni et al. 2021; Monge et al. 2021; Thompson et al. 2021; Pritchard et al. 2021) were originated from 8 countries: the United Kingdom (n = 6) (Glampson et al. 2021; Hall et al. 2021; Lopez Bernal et al. 2021; Mason et al. 2021; Pritchard et al. 2021; Shrotri et al. 2021) , the United States (n = 2) (Swift et al. 2021; Thompson et al. 2021) , Canada (n = 1) (Chung et al. 2021 ) Sweden (n = 1) (Björk et al. 2021) , Israel (n = 4) (Angel et al. 2021; Dagan et al. 2021; Haas et al. 2021; Regev-Yochay et al. 2021) , Italy (n = 1) (Fabiani et al. 2021) , Denmark (n = 1) (Emborg et al. 2021) , and Spain (n = 3) (Cabezas et al. 2021; Gras-Valentí et al. 2021; Monge et al. 2021) . Study characteristics are depicted in Table 1 . The included studies (Angel et al. 2021; Björk et al. 2021; Cabezas et al. 2021; Chung et al. 2021; Dagan et al. 2021; Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Gras-Valentí et al. 2021; Haas et al. 2021; Hall et al. 2021; Lopez Bernal et al. 2021; Mason et al. 2021; Monge et al. 2021; Pritchard et al. 2021; Regev-Yochay et al. 2021; Shrotri et al. 2021; Swift et al. 2021; Thompson et al. 2021 ) are deemed moderate-to-good quality with Newcastle-Ottawa Scale ranging from 7 to 8.

The meta-analysis of eight studies (Cabezas et al. 2021; Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Hall et al. 2021; Monge et al. 2021; Shrotri et al. 2021; Thompson et al. 2021 ) which presented effect measure as HR revealed significant protective effect against RT-PCR confirmed COVID-19 14 days or more after the first dose of BNT162b2 mRNA vaccine (pooled HR = 0.58; 95% confidence interval: 0.45-0.75; Fig. 2 ), where pooled estimate indicates vaccine effectiveness of 42% (95% confidence interval 25%-55%). Similarly, the meta-analysis of five studies (Björk et al. 2021; Dagan et al. 2021; Haas et al. 2021; Mason et al. 2021; Swift et al. 2021 ) which presented effect measure as IRR revealed significant protective effect against RT-PCR confirmed COVID-19 14 days or more after the first dose of BNT162b2 mRNA vaccine (pooled IRR = 0.47; 95% confidence interval: 0.32-0.68; Fig. 3 ), where pooled estimate indicates vaccine effectiveness of 53% (95% confidence interval 32%-68%).

Even higher vaccine effectiveness was observed 21 days or more after the first dose of BNT162b2 mRNA vaccine, where the meta-analysis of six studies (Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Hall et al. 2021; Monge et al. 2021; Shrotri et al. 2021 ) which presented effect measure as HR reported pooled HR of 0.42 (95% confidence interval: 0.31-0.57; Fig. 2 ), and thus vaccine effectiveness of 58% (95% confidence interval: 43%-69%). Likewise, the meta-analysis of three studies (Björk et al. 2021; Dagan et al. 2021; Mason et al. 2021 ) which presented effect measure as IRR reported pooled IRR of 0.41 (95% confidence interval: 0.36-0.47; Fig. 3 ), and thus vaccine effectiveness of 59% (95% confidence interval: 53-64%).

The recipient of the second dose of the BNT162b2 mRNA vaccine further boosted the vaccine effectiveness. The meta-analysis of three studies (Emborg et al. 2021 ; Haas et al. 2021; Regev-Yochay et al. 2021 ) which presented effect measure as IRR revealed significant protective effect against RT-PCR confirmed COVID-19 7 days or more after the second dose of BNT162b2 mRNA vaccine (pooled IRR = 0.09; 95% confidence interval: 0.05-0.19; Fig. 3) , where pooled estimate indicates vaccine effectiveness of 91% (95% confidence interval 81%-95%). The findings from the meta-analysis of three studies (Chung et al. 2021; Lopez Bernal et al. 2021; Pritchard et al. 2021 ) which presented effect measure as OR are also consistent (pooled OR = 0.19; 95% confidence interval 0.09-0.40) and show vaccine effectiveness of 81% (95% confidence interval 60%-91%) 7 days or more after the second dose of BNT162b2 mRNA vaccine. The meta-analysis of three studies (Hall et al. 2021; Regev-Yochay et al. 2021; Thompson et al. 2021) which presented effect measure as HR reported pooled HR of 0.12 (95% confidence interval: 0.08-0.16; Fig. 2 ) 14 days or more after the second dose, and thus vaccine effectiveness of 88% (95% confidence interval: 84%-92%). Likewise, the meta-analysis of three studies (Angel et al. 2021; Haas et al. 2021; Swift et al. 2021 ) which presented effect measure as IRR revealed significant protective effect against RT-PCR confirmed COVID-19 14 days or more after the second dose of BNT162b2 mRNA vaccine (pooled IRR = 0.04; 95% confidence interval: 0.03-0.05; Fig. 3) , where pooled estimate indicates vaccine effectiveness of 96% (95% confidence interval 95-97%).

The findings of the meta-analyses align with the phase 3 randomized controlled trial (Polack et al. 2020 ) of BNT162b2 mRNA vaccine, though with a lower protective rate: 82% after the first dose (versus overall vaccine effectiveness of 48-55% [14-21 days or more] after the first dose in the current study; Fig. 2 ) and 95% (7 days or more) after the second dose (versus overall vaccine effectiveness of 86-94% [7-14 days or more] after the second dose in the current study; Fig. 3 ). Variability in the protective rate between clinical trial and real-world studies could stem from the difference in the definition of confirmed COVID-19; confirmed COVID-19 was defined in the clinical trial as the presence of symptoms and positive RT-PCR test for SARS-CoV-2; while the included studies of our meta-analyses, confirmed COVID-19 was defined as positive RT-PCR test for SARS-CoV-2 regardless of the presence of symptoms.

In addition, individuals with comorbidities (e.g., hypertension, diabetes, and obesity) who are predisposed to severe COVID-19 constituted only about one-fifth of the study population in phase 3 randomized controlled trial (Polack et al. 2020 ) of BNT162b2 mRNA vaccine. Individuals with comorbidities (e.g., hypertension, diabetes, and obesity), especially those with old age, are often prioritized in the real-world mass vaccination campaign. Therefore, this could explain the lack of reproducible vaccine efficacy reported from the highly controlled clinical research settings compared to the real-world settings since these individuals with comorbidities mainly constituted the real-world study population. Indeed, elderly individuals with comorbidities often have diminished immune responses to vaccines (Kwetkat and Heppner 2020) .

Nevertheless, with up to 59% of real-world protective rate after the administration of the first dose of the BNT162b2 mRNA vaccine, it seems reasonable to delay the administration of the second dose in an attempt to allow vaccination in a higher proportion of individuals to reduce the risk of transmission of COVID-19 to an acceptable level. Our study was limited by the fact that included studies were originated in only a few countries. Therefore, the generalizability of our findings is unknown, especially to the countries where variants of concern of SARS-CoV-2 are circulating. Future studies should aim to investigate the vaccine effectiveness against different variants of concern of SARS-CoV-2 and with longer follow-ups to determine the duration of protection against COVID-19. Furthermore, the effectiveness of the BNT162b2 mRNA vaccine among immunocompromised individuals as well as individuals who receive treatment with immunosuppressive therapy should also be investigated since they had been excluded from the participation of phase 3 randomized controlled trial (Polack et al. 2020) . Despite its effectiveness, reporting vaccine safety data by relevant stakeholders should be encouraged as BNT162b2 mRNA is a new vaccine that has not gained full approval.

Association between vaccination with BNT162b2 and incidence of symptomatic and asymptomatic SARS-CoV-2 infections among health care workers

Effectiveness of the BNT162b2 vaccine in preventing COVID-19 in the working age population-first results from a cohort study in Southern Sweden

Effects of BNT162b2 mRNA Vaccination on COVID-19 Disease, Hospitalisation and Mortality in Nursing Homes and Healthcare Workers: a Prospective Cohort Study Including 28,594 Nursing Home Residents, 26,238 Nursing Home Staff, and 61,951 Healthcare Workers in Catalonia

Effectiveness of BNT162b2 and mRNA-1273 COVID-19 vaccines against symptomatic SARS-CoV-2 infection and severe COVID-19 outcomes in Ontario, Canada

BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting

Vaccine effectiveness of the BNT162b2 mRNA COVID-19 vaccine against RT-PCR confirmed SARS-CoV-2 infections, hospitalisations and mortality in prioritised risk groups

BioNTech/Pfizer) vaccine in preventing SARS-CoV-2 infection among healthcare workers, Treviso province, Veneto region

North West London Covid-19 Vaccination Programme: Real-world evidence for Vaccine uptake and effectiveness

Efectividad de la primera dosis de vacuna BNT162b2 para prevenir la COVID-19 en personal sanitario

Alroy-Preis S (2021) Impact and effectiveness of mRNA BNT162b2 vaccine against SARS-CoV-2 infections and COVID-19 cases, hospitalisations, and deaths following a nationwide vaccination campaign in Israel: an observational study using national surveillance data

COVID-19 vaccine coverage in health-care workers in England and effectiveness of BNT162b2 mRNA vaccine against infection (SIREN): a prospective, multicentre, cohort study

Comorbidities in the elderly and their possible influence on vaccine response

Effectiveness of the Pfizer-BioNTech and Oxford-AstraZeneca vaccines on covid-19 related symptoms, hospital admissions, and mortality in older adults in England: test negative case-control study

Effects of BNT162b2 mRNA vaccine on Covid-19 infection and hospitalisation among older people: matched case control study for England

Vaccine side-effects and SARS-CoV-2 infection after vaccination in users of the COVID Symptom Study app in the UK: a prospective observational study

Direct and indirect effectiveness of mRNA vaccination against SARS-CoV-2 infection in long-term care facilities in Spain

Vaccine effectiveness after 1st and 2nd dose of the BNT162b2 mRNA Covid-19 Vaccine in long-term care facility residents and healthcare workers-a Danish cohort study

Data (2021) Statistics and Research: Coronavirus (COVID-19) Vaccinations

Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine

Impact of vaccination on new SARS-CoV-2 infections in the United Kingdom

Decreased infectivity following BNT162b2 vaccination

Vaccine effectiveness of the first dose of ChAdOx1 nCoV-19 and BNT162b2 against SARS-CoV-2 infection in residents of long-term care facilities in England (VIVALDI): a prospective cohort study

Effectiveness of mRNA COVID-19 vaccines against SARS-CoV-2 infection in a cohort of healthcare personnel

Prevention and Attenuation of COVID-19 by BNT162b2 and mRNA-1273 Vaccines

Vaccine epidemiology: efficacy, effectiveness, and the translational research roadmap

The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.