key: cord-0011891-lumzp1ca
authors: Jackson, Michael L.; Yu, Onchee; Nelson, Jennifer C.; Naleway, Allison; Belongia, Edward A.; Baxter, Roger; Narwaney, Komal; Jacobsen, Steven J.; Shay, David K.; Jackson, Lisa A.
title: Further Evidence for Bias in Observational Studies of Influenza Vaccine Effectiveness: The 2009 Influenza A(H1N1) Pandemic
date: 2013-10-15
journal: Am J Epidemiol
DOI: 10.1093/aje/kwt124
sha: f66bf412153e5b93ae97abd39229e20d1d5fba4d
doc_id: 11891
cord_uid: lumzp1ca

Preinfluenza periods have been used to test for uncontrolled confounding in studies of influenza vaccine effectiveness, but some authors have claimed that confounding differs in preinfluenza and influenza periods. We tested this claim by comparing estimates of the vaccine-mortality association during the 2009/2010 influenza year, when there was essentially no circulation of seasonal influenza in the United States, and 2007/2008, a typical influenza year. We pooled data on seniors (adults aged ≥65 years) from 7 US managed care organizations that participated in the Vaccine Safety Datalink Project. We defined influenza vaccination, all-cause mortality, and potential confounders from administrative databases. We quantified the vaccine-mortality association using Cox regression. During 2007/2008, the adjusted hazard ratio was 0.44 prior to influenza season, 0.62 during influenza season, and 0.71 after influenza season. A similar pattern was observed during 2009/2010, when any effect of seasonal influenza vaccine observed during all time periods must have resulted from confounding: 0.65 during the autumn, 0.80 during the winter, and 0.84 during the summer. In a year with minimal seasonal influenza, we found no evidence that confounding in autumn preinfluenza periods is qualitatively different from confounding in winter. This supports the use of preinfluenza periods as control time periods in studies of influenza vaccine effectiveness.

Influenza vaccine effectiveness (VE) in seniors (adults aged ≥65 years) remains controversial (1) (2) (3) (4) (5) . Numerous observational studies have estimated that influenza vaccine reduces the risk of all-cause mortality among seniors during the winter months by 40% or more and have concluded that influenza vaccine is highly effective (4, (6) (7) (8) . In recent years, other researchers have noted that these estimates are implausibly large given the small proportion of winter deaths attributable to influenza (9) and that the apparent VE is perhaps entirely the result of uncontrolled confounding (1, 10) .

Strong evidence for confounding comes from studies that have estimated the association between influenza vaccination and risk of death during time periods before, during, and after the seasonal circulation of influenza. Time periods before and after influenza circulates are designated control time periods, during which no true vaccine benefit is expected. Studies including these control time periods have consistently found that vaccinated seniors have a significantly lower risk of death than unvaccinated seniors during all time periods, with the strongest association being observed prior to circulation of influenza viruses (1, (11) (12) (13) (14) . A highly plausible explanation for this trend is that seniors at high risk of dying are less likely to receive influenza vaccine, leading to a strong but spurious association between vaccination and reduced mortality during the months immediately following vaccination. Over time, the seniors at high risk of mortality (who are predominantly unvaccinated) die. This reduces the underlying differences in mortality risk between vaccinated and unvaccinated seniors, and the spurious association between vaccination and mortality risk attenuates towards the null. Notably, the apparent VE during influenza seasons can be fully explained by the confounding observed in the preinfluenza control time period.

Although incorporation of the preinfluenza control periods is effective and innovative, their use has been criticized by some (3, 15) . These authors have claimed that the confounding observed in autumn preinfluenza seasons is qualitatively different from confounding in winter influenza seasons. Specifically, they claim that persons who die prior to influenza season may be less likely to be offered vaccine, or may have fewer opportunities to receive vaccine, than persons who survive until influenza season. On this basis, they infer that confounding observed during preinfluenza seasons is not present when influenza circulates and that preinfluenza periods are not appropriate control time periods for studies of influenza VE. This claim could be tested by estimating the association between seasonal influenza vaccination and risk of mortality over time in a year with no seasonal influenza. If the same trends in apparent associations between vaccination and mortality were observed during a year when there could be no true vaccine effect, the claim that confounding differs between time periods would be refuted.

Circulation (16) . Thus, we hypothesized that during the entire 2009/2010 influenza year, no true association between seasonal influenza vaccine and disease outcomes should be found if confounding factors were appropriately adjusted for in multivariate analyses. We estimated the association between influenza vaccination and all-cause mortality among seniors during the 2009/2010 influenza year to see whether the confounding trend seen in prior years with typical influenza circulation remained in a year with no seasonal influenza.

We conducted a retrospective cohort study among seniors enrolled in managed care organizations (MCOs) participating in the Vaccine Safety Datalink Project, which has been described elsewhere (17 (4, 7, 18) . This set consisted of binary indicators for the presence or absence of diagnosis codes for chronic conditions in broadly defined categories (Appendix Table 1 ).

Adjusting for this "traditional" set of variables has been shown to have a negligible impact on confounding in observational VE studies (1, 19) , in part because the conditions defined within each binary variable are overly broad (10) . We also explored whether confounding could be reduced by adjusting for a second "alternate" set of more precise covariates. In addition to covariates based on ICD-9-CM codes, the alternate comorbid conditions included indicators for filled prescriptions for bronchodilators, corticosteroids, and statins (Appendix Table 1 ), based on pharmacy data from each MCO, since adjusting for prescription data has been shown to attenuate confounding in studies of influenza VE against all-cause pneumonia (19) .

Subjects were followed for 1 year from the date of study enrollment (September 1 of 2007 or 2009). The outcome of interest was all-cause mortality. Subjects were followed until death, disenrollment from the MCO, or August 31 of 2008 or 2010 (respectively), whichever was earliest.

We used Cox proportional hazards regression to estimate the hazard ratio for the association between receipt of seasonal influenza vaccine and incidence of all-cause mortality. Influenza vaccination was treated as a time-varying covariate, with individuals being classified as vaccinated 14 days after receipt of vaccine. We fitted 3 separate models for each year. In the first model, only age (as a set of categorical variables), sex, and MCO were included as model covariates. In ratio estimates were stratified by time by including an interaction term between vaccination and time period. In 2007/ 2008, we divided time into preinfluenza, influenza, and postinfluenza periods. These time periods were defined at each MCO based on regional influenza surveillance (20) . In 2009/ 2010, we divided time into pandemic (September 1-December 15), winter (December 16-March 31), and summer (April 1-August 31) periods. To further explore the change in hazard ratios over time, we estimated the vaccine-mortality hazard ratio separately for each 2-week period during each study year, adjusting for age, sex, and MCO. For the 2009/2010 influenza year, we repeated all analyses while adjusting for receipt of monovalent 2009 H1N1pdm vaccine as a time-varying covariate. The proportion of cohort members from each MCO was consistent between the two study years. Vaccinated seniors in both cohorts tended to be older than unvaccinated seniors and were more likely to have ICD-9-CM codes for heart disease, cancer, lung disease, and other chronic illnesses (Table 1) Association between vaccine and all-cause mortality During the 2007/2008 year, a year with typical influenza circulation, receipt of seasonal influenza vaccine was associated with a decreased risk of all-cause mortality after adjusting for age, sex, and MCO ( Figure 1A) . Consistent with prior studies (1, (11) (12) (13) (14) , the apparent association was strongest in the preinfluenza time period (hazard ratio = 0.44, 95% confidence interval (CI): 0.42, 0.46), decreasing to 0.62 (95% CI: 0.60, 0.65) during influenza season and to 0.71 (95% CI: 0.69, 0.74) in the summer time period. Also consistent with prior studies, adjusting for the traditional set of comorbidity indicators did not appreciably alter the estimated associations. Adjusting for the alternate comorbidity indicators also did not appreciably alter the estimated associations (Figure 1 ).

During the 2009/2010 pandemic year, receipt of seasonal influenza vaccine was also associated with a decreased risk of all-cause mortality after adjusting for age, sex, and MCO ( Figure 1B) . The apparent association was strongest during the pandemic (fall) period (hazard ratio = 0.65, 95% CI: 0.62, 0.68), decreasing to 0.80 (95% CI: 0.77, 0.83) during the winter and 0.84 (95% CI: 0.81, 0.87) during the summer.

Adjusting for the traditional or alternate comorbid conditions did not appreciably change the vaccine-mortality association. Adjusting for monovalent 2009 H1N1pdm vaccine also did not appreciably affect the results.

When hazard ratios were estimated biweekly, both study years showed similar patterns over time (Figure 2 , parts A and B). After initial instability during weeks when few doses of seasonal influenza vaccine had been given, the hazard ratio trended steadily toward the null over time.

Our findings refute the claim that confounding in preinfluenza seasons does not operate in influenza or summer seasons (3, 15) . If this claim were true, we would have observed a non-null association between influenza vaccination and mortality during the fall of 2009, followed by a null association during the winter and summer of 2009/2010. Instead, we observed a pattern in the vaccine-mortality association over time during the 2009/2010 pandemic year that was similar to the pattern observed in typical influenza seasons (1, 11, 14) , including the 2007/2008 season, which we examined in the current study. Specifically, the estimated association was highest during the preinfluenza season and trended steadily toward the null over time, but it remaining significantly protective in all time periods. Our findings are consistent with our hypothesis about confounding in observational VE studies among seniors: Differences between seniors who choose to receive vaccine and those who do not are greatest when vaccine is first made available, and the differences between vaccinated and unvaccinated groups decline slowly over time as seniors at high risk of mortality (who are predominantly unvaccinated) die, making vaccinated and unvaccinated groups more similar with respect to mortality risk over time. This study supports the use of preinfluenza periods for identifying uncontrolled confounding in observational influenza VE studies. Furthermore, this study affirms that the magnitude of uncontrolled confounding in observational VE studies is strong, as the observed confounding during preinfluenza periods is sufficient to entirely account for the apparent VE during influenza seasons.

Finally, this study affirms that adjustment for the traditional comorbidity indicators does not remove or reduce the confounding in observational studies of influenza VE against nonspecific outcomes in seniors (1, 11, 21) . Prior studies have shown that the prevalence of comorbid chronic illnesses (based on ICD-9-CM codes) is higher in vaccinated seniors than in unvaccinated seniors (1, 4, 11) . This could contribute to confounding by indication and underestimation of VE (i.e., bias toward the null) if these chronic illnesses are associated with an increased risk of death (22) . However, we have previously shown that the use of ICD-9-CM codes to define chronic illnesses can lead to substantial misclassification of health status and frailty (10) . Importantly, this misclassification is differential by influenza vaccination, and adjustment for the presence or absence of chronic illness (defined by ICD-9-CM codes) can actually increase bias in influenza VE estimates. In the present study, we found that this bias was not reduced even when we used the expanded set of ICD-9-CM comorbidity indicators. This finding suggests that increasing the specificity of potential confounders defined from ICD-9-CM codes does not capture relevant information on the difference between vaccinated and unvaccinated seniors. Available evidence suggests that detailed data on cognitive impairment, functional status, and severity of comorbid illnesses (rather than presence or absence) is necessary to substantially reduce or eliminate confounding of influenza VE estimates in seniors (19) . This study provides further evidence that outcomes such as all-cause mortality that are highly nonspecific to influenza are generally not appropriate endpoints for influenza VE estimation in seniors.

We note that the hazard ratio estimates were closer to the null during the 2009/2010 pandemic year than during the 2007/2008 influenza year. We do not believe there was a true difference between associations in these two years, as the hazard ratio estimates were further from the null during the 2007/2008 control time periods ( preinfluenza and summer, when any apparent association is the result of uncontrolled confounding) as well as during influenza season. Rather, our findings suggest that the availability of the pandemic vaccine and expected lack of benefit of seasonal influenza vaccine (Table 1) . Although the pharmacy-and ICD-9-CM-based comorbidity indicators do not capture informa-tion on the true confounders, they may indicate differences in the seniors receiving seasonal influenza vaccine between the two study years. A limitation of this study is that, in contrast to typical years, influenza viruses were circulating intensely during the fall of 2009, which could make the 2009 fall atypical with regard to death rates as compared with other preinfluenza caused little morbidity and mortality among seniors (16) , as many of these seniors had cross-protective antibodies from exposure to influenza A(H1N1) strains that circulated before 1957 (23) . Thus, the impact of the fall pandemic wave on deaths in seniors was small and should not have affected the presence of confounding in our study. Another potential limitation is that seniors could have received influenza vaccine outside of the MCO in which they were enrolled; if vaccination records were not subsequently transferred to the MCO, these seniors might have been falsely classified as unvaccinated. While previous studies conducted with the Group Health Cooperative and Northern California Kaiser populations have found that vaccine misclassification makes a negligible contribution to the bias in VE estimates (1, 24) , pandemic influenza vaccination may have been less likely to be captured in MCO electronic records than seasonal vaccination, as pandemic vaccines were more likely to be administered in non-MCO settings in some of the geographical areas in which the participating MCO provided care. When evaluated in a year with essentially no seasonal influenza circulation, estimates of the association between influenza vaccination and all-cause mortality showed a pattern of steady trend towards the null from fall through summer. Our findings are consistent with strong confounding by unmeasured factors, the strength of which declines over time. They also illustrate the usefulness of the preinfluenza time period as a control period for identifying and removing confounding in studies of influenza VE. Investigators conducting observational studies of influenza VE against nonspecific outcomes should report adjusted VE estimates during preinfluenza periods, to demonstrate the presence and strength of residual confounding.

Evidence of bias in estimates of influenza vaccine effectiveness in seniors

To rule out confounding, observational studies of influenza vaccine need to include analyses during the "preinfluenza period

Challenges in evaluating influenza vaccine effectiveness and the mortality benefits controversy

Effectiveness of influenza vaccine in the community-dwelling elderly

Mortality benefits of influenza vaccination in elderly people: an ongoing controversy

Clinical effectiveness of influenza vaccination in Manitoba

Influenza vaccination and reduction in hospitalizations for cardiac disease and stroke among the elderly

Influenza vaccination: reduction in hospitalizations and death rates among members of "Maccabi Healthcare Services" during the 2000-2001 influenza season

Impact of influenza vaccination on seasonal mortality in the US elderly population

Functional status is a confounder of the association of influenza vaccine and risk of all cause mortality in seniors

Influenza vaccination and all-cause mortality in community-dwelling elderly in Ontario, Canada, a cohort study

Influenza vaccination and mortality: differentiating vaccine effects from bias

Influenza vaccination and mortality: prospective cohort study of the elderly in a large geographical area

Influenza vaccine effectiveness in the elderly based on administrative databases: change in immunization habit as a marker for bias

Benefits of influenza vaccine in US elderly-appreciating issues of confounding bias and precision

and composition of the 2010-11 influenza vaccine

The Vaccine Safety Datalink: a model for monitoring immunization safety

Influenza vaccine effectiveness in preventing hospitalizations and deaths in persons 65 years or older in Minnesota

Influenza vaccination and risk of community-acquired pneumonia in immunocompetent elderly people: a population-based, nested casecontrol study

US Department of Health and Human Services

Why do covariates defined by International Classification of Diseases codes fail to remove confounding in pharmacoepidemiologic studies among seniors?

Confounding by indication in non-experimental evaluation of vaccine effectiveness: the example of prevention of influenza complications

Cross-reactive antibody responses to the 2009 pandemic H1N1 influenza virus

Evidence of bias in studies of influenza vaccine effectiveness in elderly patients

Evidence of Bias in Influenza Vaccine Studies 1335