key: cord-0694073-5bzavvix
authors: Roberts, E.; Gu, T.; Mukherjee, B.; Fritsche, L. G.
title: Estimating COVID-19 Vaccination Effectiveness Using Electronic Health Records of an Academic Medical Center in Michigan
date: 2022-01-31
journal: nan
DOI: 10.1101/2022.01.29.22269971
sha: c911d2ebbf33dd56cdc60961ac78f914ccb708e9
doc_id: 694073
cord_uid: 5bzavvix

Importance: Systematic characterization of the protective effect of vaccinations across time and at-risk populations is needed to inform public health guidelines and personalized interventions. Objective: To evaluate the vaccine effectiveness (VE) over time and determine differences across demographic and clinical risk factors of COVID-19. Design, Setting, and Participants: This test negative design consisted of adult patients who were tested or diagnosed for COVID-19 at Michigan Medicine in 2021. Variables extracted from Electronic Health Records included vaccination status, age, gender, race/ethnicity, comorbidities, body mass index, residential-level socioeconomic characteristics, past COVID-19 infection, being immunosuppressed, and health care worker status. Exposure: The primary exposure was vaccination status and was categorized into fully vaccinated with and without booster, partially vaccinated, or unvaccinated. Main Outcomes and Measures: The main outcomes were infection with COVID-19 (positive test or diagnosis) and having severe COVID-19, i.e., either being hospitalized or deceased. Based on these, VE was calculated by quarter, vaccine, and patient characteristics. Results: Of 170,487 COVID-19 positive adult patients, 78,002 (45.8%) were unvaccinated, and 92,485 (54.2%) were vaccinated, among which 74,060 (80.1%) were fully vaccinated. COVID-19 positivity and severity rates were substantially higher among unvaccinated (12.1% and 1.4%, respectively) compared to fully vaccinated individuals (4.7% and 0.4%, respectively). Among 7,187 individuals with a booster, only 18 (0.3%) had a severe outcome. The covariate-adjusted VE against an infection was 62.1% (95%CI 60.3-63.8%), being highest in the Q2 of 2021 (90.9% [89.5-92.1%]), lowest in Q3 (60.1% [55.9-64.0%]), and rebounding in Q4 to 68.8% [66.3-71.1%]). Similarly, VE against severe COVID-19 overall was 73.7% (69.6-77.3%) and remained high throughout 2021: 87.4% (58.1-96.3%), 92.2% (88.3-94.8%), 74.4% (64.8-81.5%) and 83.0% (78.8-86.4%), respectively. Data on fully vaccinated individuals from Q4 indicated additional protection against infection with an additional booster dose (VE-Susceptibility: 64.0% [61.1-66.7%] vs. 87.3% [85.0-89.2%]) and severe outcomes (VE-Severity: 78.8% [73.5-83.0%] vs. 94.0% [89.5-96.6%]). Comparisons between Pfizer-BioNTech and Moderna vaccines indicated similar protection against susceptibility (82.9% [80.7-84.9%] versus 88.1% [85.5-90.2%]) and severity (87.1% [80.3-91.6%]) vs. (84.9% [76.2-90.5%]) after controlling for vaccination timing and other factors. There was no significant effect modification by all the factors we examined. Conclusions and Relevance: Our findings suggest that COVID-19 vaccines offered high protection against infection and severe COVID-19, and showed decreasing effectiveness over time and improved protection with a booster.

Conclusions and Relevance: Our findings suggest that COVID-19 vaccines offered high protection against infection and severe COVID-19, and showed decreasing effectiveness over time and improved protection with a booster.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Other studies have examined the effectiveness of the vaccines in real world settings over shorter and earlier periods of time and concluded the VE is relatively high against positivity and disease severity such as hospitalization and death [2] [3] [4] [5] [6] [7] [8] [9] . While these studies support evidence of robust protection against variants with modest waning of VE through the third quarter of 2021, the FDA approved booster shots for continued protection of the vaccinated population over time.

Still, the time-varying protection of vaccination over the course of 2021 and effectiveness among different vaccine brands remain unclear. More data from healthcare centers is needed to track hospitalizations and deaths across vaccination status. It is important to adjust for covariates that may be related to both getting vaccinated and severe COVID-19 outcomes as potential confounders in real-world studies 10 . Electronic health records (EHRs) are helpful for this type of research as they allow for such big data and covariate adjustment.

The objective of this work is to evaluate VE and determine if any patient characteristics show effect modification of VE using EHR from the University of Michigan Health System, in . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted January 31, 2022. series or did not adhere to the guidelines and received a mixed sequence of the abovementioned vaccines ( Table 1, Table S1 ). 254 individuals who received unspecified or other . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted January 31, 2022. ; https://doi.org/10.1101/2022.01.29.22269971 doi: medRxiv preprint brand vaccines (AstraZeneca, Sinopharm, Sinovac, or Novavac) were not included in the study sample. Being boosted was defined as receiving any additional vaccination any time after the original full vaccination schedule.

In this study, we focused on two COVID-19 associated outcomes: susceptibility, i.e., tested positive / diagnosed for COVID-19, and severity, i.e., having severe COVID-19 defined as either being hospitalized, admitted to an intensive care unit (ICU) within one month after a positive COVID-19 test / diagnosis, or died within two months after a positive COVID-19 test / diagnosis ( Table 2 ). The summary of these outcomes is listed in Table 2 , stratified by vaccination status.

We extracted the EHR data from patients with COVID-19 diagnoses or test results for COVID-19 (positive or negative) at MM, including age, self-reported gender, self-reported race/ethnicity, body mass index (BMI; calculated as weight in kilograms divided by height in meters squared), Neighborhood Socioeconomic Disadvantage Index (NDI) without proportion of Black (coded as quartiles, with larger quartiles representing more disadvantaged communities), and population density measured in persons per square mile (coded as quartiles). The Elixhauser comorbidity score developed by the Agency for Healthcare Research and Quality (AHRQ) was calculated to comprehensively characterize patients' pre-existing comorbidity conditions using ICD9 and ICD10 codes and the R package "comorbidity" 11 . Past COVID-19 infection was defined as testing positive at least 21 days before the first test of a quarter (if a person remained negative in that quarter), or 21 days before the first positive test of a quarter (if tested positive at least once in that quarter). Being immunosuppressed was defined for patients . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted January 31, 2022. ; https://doi.org/10.1101/2022.01.29.22269971 doi: medRxiv preprint who had any immunosuppressants listed in their medication or prescription records within one

year before a corresponding test (Calcineurin inhibitors, Interleukin inhibitors, TNF-alfa inhibitors, or any selective or other immunosuppressants) 12 . In addition, health care worker status was defined based on documented participation in a Health Care Worker Screen or a SARS-CoV-2 PCR test order for Health Care Workers. We report the variables' missingness by vaccination status in Table S2 . Missing data were handled by using a complete case analysis.

We fit Firth's bias-corrected logistic regression for each COVID-19 related outcome

, considering several sets of covariates:

is the indicator of vaccination status, either with or without a booster, i.e. We estimated the vaccine efficacy (VE) through the following formula 13 :

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 31, 2022. ;

, the ratio of incidence between the vaccinated group and the unvaccinated group, can be estimated using the Michigan 12-month COVID-19 incidence rate (IR) and the estimated odds ratio (OR) from above logistic regression through the following formula 14 :

We To assess the potentially different VE across different strata of risk factors (denoted as X) associated with COVID-19 outcomes, we further conducted interaction analysis by vaccine status using the following model:

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 31, 2022. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 31, 2022. ; https://doi.org/10.1101/2022.01.29.22269971 doi: medRxiv preprint Observed Outcomes When assessing COVID-19 positivity in the overall cohort, we observed substantially higher rates in unvaccinated (12.1%) compared to partially vaccinated (6.0%) or fully vaccinated individuals (4.7%). The positivity rate among fully vaccinated individuals reached a valley with low community transmission in Q2 (Q1: 1.3% and Q2: 1.1%) and surged to 3.2% in Q3 and to 6.7% in Q4 ( Table 2) Table 2) .

To next estimate the VE of a full vaccination against an infection (VE-Susceptibility) and against severe COVID-19 (VE-Severity), we applied logistic regression models unadjusted and adjusted for relevant covariates (see Methods, Table S3 ). To reduce bias in the estimates, it is important to adjust for potential confounding factors that may be related to both getting vaccinated (our exposure) and outcome (COVID positivity and severity) which include age, gender, race/ethnicity, comorbidity score, residential-level socioeconomic characteristics, past COVID-19 Infection, and health care worker status. In Figure 1 , we present VE estimates for . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 31, 2022. ; https://doi.org/10.1101/2022.01.29.22269971 doi: medRxiv preprint the overall and the time-stratified cohort (Q1, Q2, Q3, and Q4). Since the VE estimates were reasonably consistent across adjustments, e.g., in the overall cohort VE-Susceptibility ranged between 59.4 and 62.1%, we will only discuss the estimates from the most conservative setting Figure S2B) indicating that the VE estimates in the full cohort might be influenced by incomplete vaccination documentation in the EHR (Figure S2) .

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 31, 2022. ; https://doi.org/10.1101/2022.01.29.22269971 doi: medRxiv preprint

Since the majority of the 74,060 fully vaccinated individuals completed a vaccine series with either Pfizer-BioNTech (n = 45,168, 61.0%) or Moderna (n = 25,267, 34.1%), we next compared the effectiveness of the two vaccines. To reduce the differing impact of a waning effectiveness, i.e., the Pfizer-BioNTech vaccine was available earlier than the Moderna vaccine, we focus on the analysis which only included individuals who reached full vaccination status less than 3 months before their COVID-19 test or diagnosis (Figure 2) .

In the overall cohort, we observed lower VE-Susceptibility for Pfizer-BioNTech compared is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 31, 2022. ; https://doi.org/10.1101/2022.01.29.22269971 doi: medRxiv preprint acquired immunity through a previous COVID-19 infection likely do not benefit as much from a vaccination as people who were not previously infected. It is important to note that only 182 individuals were reported as being reinfected in the data, and only 48 of these were fully vaccinated. 

This study of 170,487 adults tested/diagnosed for COVID-19 at MM provides additional evidence of high COVID-19 vaccine effectiveness against SARS-CoV-2 variants across 2021. In our study, fully vaccinated individuals were more likely to test negative and were less likely to have severe disease than unvaccinated individuals or those with unknown vaccination status.

Qualitatively, we observed waning effects of VE against positivity and disease severity across the four quarters. However, as the number of individuals who received booster shots increased in the fourth quarter of 2021, both infection and severity rates were lowest among boosted . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 31, 2022. Our results are largely consistent with previously published findings, though differences among studies make direct comparison challenging, e.g., some studies adjusted for potential confounders (age only, other patient characteristics, or time since vaccination) while others characterized the variant (either Alpha, Mu, or Delta) associated with each case via wholegenome sequencing. Our study spans the whole year of 2021 and thus likely includes data on multiple variants (including the Omicron variant) and also on booster doses that have become more prevalent. From other studies conducted in late 2020 to early 2021, VE estimates against hospitalization ranged around 87% (CI 55-100%) 2 and 89% (CI 87-91%) 3 CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 31, 2022. A strength of the present study is the availability of detailed EHR data for study subjects.

This allows us to adjust for potential confounding patient characteristics including socioeconomic and demographic characteristics and relevant medical conditions including previous COVID-19 testing results and receipt of a booster dose. Most often, the test negative design has been used to evaluate VE 3,22,23 . As such, a limitation of results from this study design is the generalizability to individuals tested for COVID-19. Still, these studies provide insight about the long-term effectiveness and durability of vaccine protection 23 . A potential weakness is any missing information from the EHR, e.g., undocumented test results or vaccinations, though we believe categorizing individuals into an unknown/unvaccinated status tends to produce conservative VE estimates. Overall, these results should give confidence in the effectiveness of the vaccines and may encourage those who have not been vaccinated or not received a booster to do so.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 31, 2022. infections to protect healthcare workers from variants of concern: A 14-month observational study using surveillance data. PLoS One. 2021;16(7):e0254920.

Simione L, Gnagnarella C. Differences Between Health Workers and General . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 31, 2022. ; is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 31, 2022. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 31, 2022. ; Table 2 . COVID-19 Outcome Summary of 170,487 individuals who were tested for or diagnosed with COVID-19 between January and December 2021. For individuals who were tested/diagnosed across multiple quarters, the vaccination status at their last observed quarter is summarized in the overall cohort. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted January 31, 2022. 1.87 (1.61, 2.17) 7.1e−17

Reference 1.26 (1.08, 1.47)  0.0038  1.37 (1.17, 1.61) 7.1e−05 1.63 (1.38, 1.94 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted January 31, 2022. ; https://doi.org/10.1101/2022.01.29.22269971 doi: medRxiv preprint

CDC

BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Mass Vaccination Setting

Vaccines in Ambulatory and Inpatient Care Settings

Effectiveness of mRNA-1273 against delta, mu, and other emerging variants of SARS-CoV-2: test negative case-control study

Sustained Effectiveness of Pfizer-BioNTech and Moderna Vaccines Against COVID-19 Associated Hospitalizations Among Adults -United States

Effectiveness of COVID-19 mRNA Vaccines Against COVID-19-Associated Hospitalization -Five Veterans Affairs Medical Centers, United States

New COVID-19 Cases and Hospitalizations Among Adults, by Vaccination Status

VE (95% CI) 59.8% (57.2%, 62.3%)

VE (95% CI)