key: cord-1024849-nlll8fay
authors: Feng, Shuo; Cowling, Benjamin J; Kelly, Heath; Sullivan, Sheena G
title: Estimating Influenza Vaccine Effectiveness With the Test-Negative Design Using Alternative Control Groups: A Systematic Review and Meta-Analysis
date: 2017-06-21
journal: American Journal of Epidemiology
DOI: 10.1093/aje/kwx251
sha: c9ff2403f8870c64dbb5b6c7ee549d0e94f77257
doc_id: 1024849
cord_uid: nlll8fay

One important assumption in case-control studies is that control selection should be independent of exposure. Nevertheless, it has been hypothesized that virus interference might lead to a correlation between receipt of influenza vaccination and increased risk of infection with other respiratory viruses. We investigated whether such a phenomenon might affect a study design commonly used to estimate influenza vaccine effectiveness (VE). We searched publications in MEDLINE, PubMed, and Web of Science. We identified 12 studies using the test-negative design (2011–2017) that reported VE estimates separately derived by 3 alternative control groups: 1) all patients testing negative for influenza (FLU), VE(FLU−); 2) patients who tested positive for other/another respiratory virus (ORV), VE(ORV+); and 3) patients who tested negative for all viruses in the panel (PAN), VE(PAN−). These included VE estimates from 7 countries for all age groups from 2003/2004 to 2013/2014. We observed no difference in vaccination coverage between the ORV-positive and PAN-negative control groups. A total of 63 VE(FLU−) estimates, 62 VE(ORV+) estimates, and 33 VE(PAN−) estimates were extracted. Pooled estimates of the difference in VE (ΔVE) were very similar between groups. In meta-regression, no association was found between the selection of control group and VE estimates. In conclusion, we did not find any differences in VE estimates based on the choice of control group.

Initially submitted January 26, 2017; accepted for publication June 9, 2017.

One important assumption in case-control studies is that control selection should be independent of exposure. Nevertheless, it has been hypothesized that virus interference might lead to a correlation between receipt of influenza vaccination and increased risk of infection with other respiratory viruses. We investigated whether such a phenomenon might affect a study design commonly used to estimate influenza vaccine effectiveness (VE). We searched publications in MEDLINE, PubMed, and Web of Science. We identified 12 studies using the test-negative design (2011-2017) that reported VE estimates separately derived by 3 alternative control groups: 1) all patients testing negative for influenza (FLU), VE FLU− ; 2) patients who tested positive for other/another respiratory virus (ORV), VE ORV+ ; and 3) patients who tested negative for all viruses in the panel (PAN), VE PAN− . These included VE estimates from 7 countries for all age groups from 2003/2004 to 2013/2014. We observed no difference in vaccination coverage between the ORV-positive and PAN-negative control groups. A total of 63 VE FLU− estimates, 62 VE ORV+ estimates, and 33 VE PAN− estimates were extracted. Pooled estimates of the difference in VE (ΔVE) were very similar between groups. In meta-regression, no association was found between the selection of control group and VE estimates. In conclusion, we did not find any differences in VE estimates based on the choice of control group. epidemiologic methods; influenza; test-negative design; vaccine effectiveness; virus interference Abbreviations: CI, confidence interval; ED, emergency department; ORV, other/another respiratory virus; PCR, polymerase chain reaction; RSV, respiratory syncytial virus; TND, test-negative design; VE, vaccine effectiveness.

Influenza vaccines are the most effective measure available for reducing the substantial annual disease burden associated with Influenzavirus infections. Influenza vaccines generally have moderate-to-good efficacy, estimated to fall within the range of 60%-70% in children and 50%-60% in adults based on randomized controlled trials (1, 2) . However, influenza vaccine efficacy and vaccine effectiveness (VE) can vary from year to year depending on the degree of antigenic match between strains selected for inclusion in the vaccine and circulating strains, as well as the intervals between vaccination and influenza epidemics (3) . VE can also vary among subpopulations-persons of different ages, for example (4, 5) . These factors may affect both experimental and observational study designs. Thus, annual estimates of influenza VE can provide ongoing evidence on the performance of influenza vaccines in the community.

Although randomized controlled trials are considered the gold standard for measuring vaccine efficacy, for the purpose of making annual VE estimates, observational studies are carried out more commonly because of feasibility, efficiency, and ethics. The test-negative design (TND) has been widely applied for measuring influenza VE on a routine basis in Europe, North America, Australia, and Asia since 2005 (6, 7) . In this study design, patients with signs and symptoms meeting predefined clinical definitions (e.g., acute respiratory infection or influenzalike illness) are swabbed and tested for influenza viruses. In some cases, testing may be done against a panel of respiratory viruses. Persons testing positive are defined as cases, while those testing negative for influenza viruses (influenza (FLU)-negative) are classified as controls. Persons who test negative may test positive for another respiratory virus (other/another respiratory virus (ORV)-positive) or may test negative for all viruses in the panel (panel (PAN)-negative). VE is estimated as VE% = 1 − OR adj , where OR adj is the odds ratio comparing the odds of vaccination among cases with the odds of vaccination among the controls, adjusted for potential confounders such as age. This design can be embedded within existing surveillance systems to enable timely estimation of VE in both inpatient and outpatient settings at a reasonable cost (8, 9) .

Although the number of studies using the TND has been increasing, the theoretical underpinnings and inherent assumptions of the TND need further evaluation (10, 11) . As a type of case-control study, the TND should follow the basic principles of control selection: 1) controls should be selected from the same population as cases and 2) controls should be selected independently of exposure, within each stratum of factors included in stratified analysis (12) . By restricting cases and controls attending outpatient clinics or hospitals to the same clinical case definition (e.g., influenza-like illness (9)), the TND includes controls from the same source population as cases, and thus reduces bias from differential health-care-seeking behavior.

However, TND studies conducted in inpatient settings could be complicated by acute exacerbation of chronic respiratory conditions, which could be difficult to distinguish from respiratory virus infections requiring hospitalization. Furthermore, if this population were more likely to receive influenza vaccination than the source population, selection bias would be introduced. Nevertheless, in a meta-analysis comparing inpatient and outpatient VE, Feng et al. (13) observed similar VE estimates by setting when the analysis was restricted to the same season, country, and age group. In a simulation study examining the validity of the TND for inpatient settings, Foppa et al. (14) concluded that VE estimates would be biased if chronic respiratory conditions were not well-controlled in the analysis. Under this assumption, vaccination coverage and VE estimates might differ according to different control groups; that is, higher vaccination coverage, and thus higher VE estimates, would be observed with the use of PAN-negative controls.

An assumption of case-control studies is that control selection should be independent of exposure. For influenza VE studies, this assumption may be violated if the risk of infection with a noninfluenza virus is not independent of vaccination status (15, 16) . In a randomized controlled trial, Cowling et al. (15) observed increased risk of ORV infection among 2008-2009 trivalent influenza vaccination recipients. Several epidemiologic studies have observed "viral interference," a phenomenon in which infection by one virus alters susceptibility to infection by another virus. This has been reported, for example, for influenzavirus and rhinovirus (17) and influenzavirus and respiratory syncytial virus (RSV) (18) .

The purported underlying mechanism behind these phenomena might involve both nonspecific immunity and influenzavirusspecific immunity (19) . After a viral infection, temporary nonspecific immunity against a second infection is induced, which could last for several weeks (20) . Given its brief duration, investigators might be unlikely to observe a population or cohort effect. In contrast, the influenzavirus-specific interference may involve T-cell-mediated immunity, which may last for months because of broad heterotypic cross-reactivity. The immune response might vary across different influenza strains and could possibly be more evident during an influenza pandemic wherein specific influenza strains predominate during the season (19) . Although the nonspecific and specific immunity against influenza and other infectious diseases has been broadly described, the potential nature and biological mechanisms of virus interference remain unclear (21) .

For the TND, virus interference may be important if, by preventing influenza infection through vaccination, nonspecific and influenzavirus-specific immunity is not induced and leaves the vaccinee susceptible to infection by cocirculating viruses he or she might otherwise have been protected from. This would result in higher influenza vaccination coverage among controls with ORVs detected than in controls with no virus detected. Therefore, selection of controls could then lead to biased estimation of VE; that is, the VE estimates derived using the ORV-positive group would be higher than those for the PAN-negative group and the FLU-negative group. Simulation studies have suggested that while this phenomenon can produce biased estimates, that bias is trivial except under extreme conditions (22) . However, simulations are often simplistic representations of real studies and may not be able to capture the nuances of immunological phenomena. For example, immunological responses may be less apparent in the elderly than in children or young adults, possibly because of immunosenescence (23) . Thus, any evaluation of the potential bias caused by virus interference should consider the age of the population.

In this review, we aimed to assess whether virus interference could affect VE estimates generated from studies using the TND. First, we compared vaccination coverage using alternative control groups. Second, we compared VE estimates according to the choice of control group. Third, we summarized the total difference in VE estimates and assessed deviations from zero. Finally, we assessed whether any differences in VE were equally apparent among all age groups and in different study settings.

We previously reviewed TND studies that estimated influenza VE (6, 8, 13) . For the current review, we reassessed all papers derived from our previous searches (last performed on December 28, 2015 (13)) and conducted an online update on April 18, 2017. Following the previous search strategies, papers were searched on PubMed (National Library of Medicine, Bethesda, Maryland), MEDLINE (National Library of Medicine), and Web of Science (Clarivate Analytics, Philadelphia, Pennsylvania) for the following combination of keywords: 1) "influenza" or "flu"; 2) "vaccine effectiveness" or "VE"; 3) "test-negative" or "test negative" or "case-control" or "case control"; and 4) sets 1, 2, and 3. Articles were independently screened by 2 of the authors (S.F. and S.G.S.). Studies estimating influenza VE for any season, any influenza type/subtype, or any type of influenza vaccination by VE were considered. Only articles published in English were considered. Studies or subanalyses of studies which reanalyzed previously published data, reported interim estimates, or did not use the TND were excluded. All studies meeting these inclusion criteria were further screened and were included if influenza VE was estimated using alternative control groups, including FLU-negative, ORV-positive, and/ or PAN-negative controls.

Study design information was extracted for each included study using a standardized form. This included: author, publication year, study country, influenza season, population ages, study setting, surveillance system, case definition, time intervals since symptom onset, type of swab, laboratory methods, ORVs tested, vaccination coverage in each control group, covariates included in statistical models, and all VE estimates using alternative control groups.

Differences in influenza vaccination coverage and estimates of VE against influenza A or B according to alternative control groups were compared by paired t test. Adjusted VE estimates obtained using all available control groups were extracted, including the FLU-negative group (denoted VE FLU− ), the ORVpositive group (denoted VE ORV+ ), and the PAN-negative group (denoted VE PAN− ). Stratified VE estimates were also extracted by influenza type/subtype for each age group, influenza season, and setting (inpatient or outpatient). For each study, the differences in available VE estimates (ΔVE) were defined as:

We calculated 95% confidence intervals for each ΔVE by bootstrap, using 1,000 resamples (see the Web Appendix, available at https://academic.oup.com/aje). We excluded studies with large uncertainty, as defined by a 95% confidence interval range for VE FLU− of more than 100 percentage points. The remaining studies were pooled, and estimates of ΔVE FLU−,ORV+ , ΔVE FLU−,PAN− , and ΔVE ORV+,PAN− against influenza A or B were calculated. Where studies provided both overall estimates and estimates for subgroups, we removed the overall estimate to avoid any overlap. We performed Egger's test on ΔVE FLU−,ORV+ and its standard error to assess publication bias. A fixed-effects model was assumed, and heterogeneity was examined by means of the I 2 statistic and Cochran's Q test. The inverse of the variance of ΔVE FLU−,ORV+ was used to weight the studies.

To test our hypothesis that virus interference may vary by age, we further estimated ΔVE FLU−,ORV+ by age group-that is, children (ages 6 months-17 years), young adults (ages 18-49 years), and older adults (ages ≥50 years). Since inpatient studies may be biased due to recruitment of patients with chronic respiratory disease rather than viral infection, we also examined ΔVE by setting, whenever possible. We evaluated whether pooled ΔVE FLU−,ORV+ differed from zero among children or young adults more than among elderly adults. To further examine whether any study design feature was associated with VE estimates, we also conducted meta-regression by means of univariate and multivariate random-effects models. The predictors included age group (children, adults, elderly, and all ages), study setting (inpatient/outpatient/mixed), season (single/multiple), restriction of patients to those presenting for health care within 4 days (yes/no), exclusive use of polymerase chain reaction (PCR)/reverse transcription PCR for testing (yes/no), number of ORVs tested, and type of control group (FLU-negative, ORV-positive, or PAN-negative). All analyses were conducted using R, version 3.3.1 (R Foundation for Statistical Computing, Vienna, Austria) and the metafor package.

A total of 120 publications were retrieved from the previous study (6, 8, 13) , and 35 articles were obtained from the new search. Based on 155 test-negative studies, 12 articles that met the inclusion criteria were identified (24-35) (Figure 1 ). Two studies reported estimates from the same population (24, 27) . Kelly The first study estimating VE using alternative control groups was a 2011 study from Australia (27) . Study characteristics are summarized in Table 1 . The studies covered the 2003/2004 to 2013/2014 influenza seasons and 7 countries, including Australia (4 studies (24, 27, 28, 31) ), the United States (3 studies (26, 32, 34) ), Japan (1 study (33)), China (Hong Kong; 1 study (25)), Portugal (1 study (29)), New Zealand (1 study (30) ), and the Netherlands (1 study (35)) ( Table 1) . Five studies reported estimates for a single year (27, (29) (30) (31) 33) , while others reported estimates across multiple years (24-26, 28, 32, 34, 35) . Among 7 studies reporting pooled estimates across more than 1 year/season, 3 studies also provided season-specific estimates (28, 34, 35) .

Patients of all ages were investigated in 7 studies (26, 28-31, 34, 35) , children in 4 studies (24, 25, 27, 32) , and children, adults, and elderly persons in 2 studies (30, 34) . Eight studies were carried out exclusively in outpatient settings (24, 27-29, 31, 33-35) , including 1 in an ED (24); 2 were carried out in inpatient settings (25, 26) ; 1 used both inpatients and outpatients, with estimates broken down by setting (30) ; and 1 recruited outpatients (including ED patients), urgent-care patients, and acutecare patients (32) . Various case definitions were identified. Some studies used an influenza-like illness definition (n = 6 studies), others an acute respiratory infection definition (n = 3); one used either an influenza-like illness or acute respiratory infection definition, while another required hospitalized pneumonia (n = 1); and 1 Japanese study required use of rapid influenza detection tests. The case definitions used are summarized in Web Table 1 . All studies included fever or history of fever in the clinical case definition, with the exception of 1 study which enrolled hospitalized pneumonia patients (26) . Various restrictions were applied to the inclusion of patients based on the interval between symptom onset and clinical presentation, including presentation within 4 days, 7 days, and 10 days; 2 studies did not specify such a restriction (Table 1) .

All studies estimated VE using both FLU-negative and ORV-positive controls. Seven also reported VE estimates obtained using PAN-negative controls (25, 26, 29, (32) (33) (34) (35) . All studies performed PCR/reverse transcription PCR for influenza diagnosis among all patients or in a subset of patients. Some studies also used immunofluorescence and/or virus culture (24, 25, 27, 28) (Table 1 ). The number of ORVs included in the respiratory panels ranged from 3 to 8. All included studies tested for RSV. Other commonly included viruses were human metapneumovirus (n = 9 studies), adenovirus (n = 8), and rhinovirus (n = 9) ( Table 1) . Only 2 studies reported the number of specimens testing positive for each ORV tested (33, 34) . In these 2 studies, the most commonly detected ORVs were RSV and rhinovirus, accounting for 66.5% (34) and 73.1% (33) of total ORVs detected, respectively. There was a weak negative correlation between the number of ORVs included in the panel and the proportion of patients identified as PAN-negative (r = -0.37), but this was not statistically significant (P = 0.13). The proportion of controls identified as PAN-negative was lowest among children: 17.3% in 1 US study (32), 26.8% in 1 Australian study conducted in an ED (24) , and 21.3% from another Australian study conducted in an outpatient setting (27) ( Table 2) . The proportion of patients testing positive for ORVs in the FLUnegative group ranged from 13.5% among people aged ≥10 years in Japan (33) to 80.2% among children aged 6-59 months in the United States (32) . The interquartile range was 27.5%-45.5% for all studies (Table 2) .

Vaccine coverage was reported by all studies for the various control groups considered. In addition, vaccine coverage by age group and control group was calculated for each study, where sufficient detail was provided. Because Pierse et al. (30) reported results by study setting (inpatient and outpatient), Feng et al. (34) and Levy et al. (28) reported results by season, and Sundaram et al. (32) reported results by age group, we were able to calculate estimates for a total of 18 triplets from 12 studies ( Table 2 ). The vaccination coverage estimated from the FLU-negative group ranged from 9.0% among inpatient children aged 6 months-17 years in Hong Kong, China, in 2009-2013 (25) to 68.3% among outpatient children aged 6 months-5 years in Australia in 2008 (27) . The differences in vaccination coverage between the ORV-positive and PAN-negative groups ranged from −28.4% to 16.0% (Table 2 ). However, the paired t test comparing mean vaccination coverage between these groups suggested no statistical difference (P = 0.61). Of 18 differences in vaccination coverage point estimates, 6 were higher than 10%. We did not identify any age-or setting-specific pattern larger or smaller than 10% by univariate analysis.

We extracted 63 VE FLU− estimates, 62 VE ORV+ estimates, and 33 VE PAN− estimates for further comparison. The distributions of VE FLU− and VE ORV+ estimates are shown in Figure 2A , while the distributions of VE FLU− and VE PAN− estimates are shown in Figure 2B . We observed VE estimates to be correlated and mostly distributed close to the identity lines. Among the 63 VE FLU− estimates, 38 estimates were for persons of all ages, 18 were for children, and 4 were specifically for older adults; 44 were estimates of VE against influenza A or B viruses, while others were calculated for a specific influenza type/ subtype (Web Figures 1 and 2) . All studies estimated VE after adjusting for potential confounders, including age, and 9 in 12 adjusted for calendar time as a proxy for changing influenza activity. We calculated ΔVE FLU−,ORV+ , ΔVE FLU−,PAN− , and ΔVE ORV+,PAN− from each available estimate. We did not find evidence of publication bias by assessing ΔVE FLU−,ORV+ and its standard error (P = 0.64). No statistically significant differences were observed for any of the ΔVE estimates except those reported by van Doorn et al. (35) .

After restriction of the VE FLU− estimates analyzed to those with confidence intervals spanning 100 percentage points or less, 48 VE FLU− estimates, 48 VE ORV+ estimates, and 24 VE PAN− estimates remained for further analysis (Web Figure 1) . The 48 point estimates of ΔVE FLU−,ORV+ ranged from -43% to 18%. Forty of these estimates had an absolute value of less than 10% (Web Figure 1) . The VE estimates excluded from meta-analysis are shown in Web Figure 2 . In the meta-analysis, we removed pooled ΔVE estimates if ΔVE estimates by age stratum/influenza season were available (n = 21) and estimated pooled ΔVE FLU−,ORV+ against influenza A or B viruses. I 2 and Cochran's Q test implied no concerning heterogeneity (I 2 = 0, were not included. e There were 4 subjects in the ORV-positive control group and 18 subjects in the PAN-negative control group with missing data on vaccination status. Persons with missing vaccination status were not excluded; their vaccination status was categorized as "unknown," and they were included in the analysis. Q(20 df) = 8.4, P = 0.99). The pooled ΔVE FLU−,ORV+ was -4% (95% confidence interval (CI): -10, 2) as estimated from 21 pairs of differences in VE estimates (ΔVE FLU− and ΔVE ORV+ ), consistent with no substantial difference for VE estimates between the FLU-negative and ORV-positive groups. Similarly, the pooled estimate for ΔVE FLU−,PAN− was -1% (95% CI: -8, 5) (n = 13), and that for ΔVE ORV+,PAN− was 5% (95% CI: -2, 12) (n = 13). Web Figure 2 shows the confidence intervals of VE FLU− estimates spanning over 100 percentage points.

We further conducted meta-analysis on ΔVE FLU−,ORV+ estimates for children (7 ΔVE estimates), adults (3 ΔVE estimates), and elderly (2 ΔVE estimates). The pooled ΔVE FLU−,ORV+ was 0% (95% CI: -9, 8) for children, -4% (95% CI: -21, 14) for young adults, and 1% (95% CI: -20, 22) for older adults. We were not able to identify any trend among age groups. We also performed sensitivity analysis by restricting VE estimates to specific influenza types/subtypes (H1N1, H3N2, and B) and settings (inpatient or outpatient). The pooled estimate was -2% (95% CI: -19, 15) from 3 ΔVE FLU−,ORV+ estimates for inpatient settings and -6% (95% CI: -13, 1) from 17 ΔVE FLU−,ORV+ estimates for outpatient settings. We did not observe a pooled ΔVE FLU−,ORV+ that was statistically different from zero.

We performed meta-regression to explore factors that may contribute to VE using univariate and multivariate randomeffects models. In univariate models, VE estimates were higher if the estimate was for children (P = 0.001), persons of all ages (P < 0.001), or inpatients (P = 0.04) (Web Table 2 ). No association was observed between VE estimates and choice of control group. Similarly, we did not observe an interaction between age group and selection of control group (P > 0.05; results not shown). In the multivariate regression, we also did not observe a significant association between the choice of control group and VE (P (ORV+) = 0.24, P (PAN−) = 0.93) (Web Table 2 ).

Based on 12 studies identified from 155 TND publications providing 63 VE FLU− estimates, 62 VE ORV+ estimates, and 33 VE PAN− estimates, we did not find any statistical differences in VE by type of control group. The paired t test assessing vaccination coverage also did not demonstrate any difference by choice of control group. Although we observed a ΔVE FLU−,ORV+ less than zero reported in 1 study, the difference was not consistently observed in each season (35) . The pooled ΔVE FLU−,ORV+ estimate for 21 pairs of observations was -4% (95% CI: -10, 2), suggesting that the choice of control group is unlikely to significantly affect VE estimated using the TND.

The nonspecific immune response to influenzavirus involves the activation of natural killer cells, macrophages, and dendritic cells, the immune functions of which are known to decrease with age (23) . Therefore, we assumed that the phenomenon of virus interference may be more prevalent among younger, rather than older, age groups. However, our analysis did not strongly support this hypothesis; although age group was correlated with VE estimates, we did not observe an interaction between any age groups and choices of control groups in multivariate metaregression. We acknowledge that our sample size was small, and as the number of studies examining this phenomenon increases, evidence for virus interference may arise.

Virus interference may act differently by type/subtype. For example, interference between RSV and influenza A may be more pronounced than interference between RSV and influenza B (36; Dr. Karen Laurie, WHO Collaborating Center for Reference and Research on Influenza at the Peter Doherty Institute for Infection and Immunity, Melbourne, Victoria, Australia, personal communication, 2016). However, we identified no statistically significant ΔVE estimates among types/subtypes. Again, our analysis of these effects was limited by the sample size. Selection bias may arise from inpatient studies if patients hospitalized due to chronic underlying conditions other than infection are also more likely to be vaccinated. In this study, we identified 3 studies reporting inpatient results and did not observe any such trend. The pooled VE estimates by setting did not differ from zero.

We found that the proportion of PAN-negative controls varied substantially among studies. These differences could partially be explained by the number of ORVs tested. However, we only observed a nonsignificant, weak, negative correlation between the number of ORVs tested and the proportion PANnegative. The types of viruses included in the panel may also affect the proportion PAN-negative. For example, in the 2 studies reporting the detection rates of each ORV, rhinovirus and RSV were most commonly detected. Only 1 study from Hong Kong (25) did not include rhinovirus, which could explain why this study had the highest proportion PAN-negative. The differences in proportion PAN-negative are likely to also be associated with viral load and shedding. With the exception of the Hong Kong study (25) , the studies involving children presented low proportions of PAN-negative subjects among all eligible subjects, which is consistent with observations of higher viral load and shedding from children (37) . The heterogeneity of viral shedding by age suggests potential misclassification bias in the PAN-negative group, where results may be more likely to be false-negative among samples taken from older patients. Other reasons for false-negative influenza results may be associated with suboptimal swab quality, imperfect laboratory testing methods, or long intervals from symptom onset to presentation (11) . All of these factors could contribute to misclassification of infection status and contribute to larger ΔVE ORV+,PAN− values. Under this circumstance, the ORV-positive group could be considered the one providing more accurate VE estimates, because it demonstrates that the swab and swabbing method were of sufficient quality to detect virus (28) .

In conclusion, based on 12 studies estimating VE using alternative control groups, we did not find evidence of virus interference, suggesting that VE estimates obtained by means of the TND are not biased by virus interference. Using FLU-negative controls is likely to produce VE estimates that are as reliable as those of ORV-positive and PAN-negative controls. From a resource-saving perspective, investigators in surveillance systems or research schemes using the TND to measure influenza VE may consider not testing further for ORVs when estimating influenza VE. Further simulation studies that incorporate multiple sources of bias and examine this phenomenon in different age groups could help confirm or refute our findings.

Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysis

Assessment of the efficacy and effectiveness of influenza vaccines in healthy children: systematic review

Comparing influenza vaccine efficacy against mismatched and matched strains: a systematic review and meta-analysis

Enhanced genetic characterization of influenza A(H3N2) viruses and vaccine effectiveness by genetic group

Early influenza vaccine effectiveness results 2015-16: I-MOVE multicentre casecontrol study

Potential of the test-negative design for measuring influenza vaccine effectiveness: a systematic review

Estimation of influenza vaccine effectiveness from routine surveillance data

Concordance of interim and final estimates of influenza vaccine effectiveness: a systematic review

The test-negative design for estimating influenza vaccine effectiveness

Observational studies and the difficult quest for causality: lessons from vaccine effectiveness and impact studies

Theoretical basis of the test-negative study design for assessment of influenza vaccine effectiveness

Case-control studies

Influenza vaccine effectiveness by test-negative design-comparison of inpatient and outpatient settings

The case testnegative design for studies of the effectiveness of influenza vaccine in inpatient settings

Increased risk of noninfluenza respiratory virus infections associated with receipt of inactivated influenza vaccine

Virus interference and estimates of influenza vaccine effectiveness from test-negative studies

Virus interference. Did rhinoviruses activity hamper the progress of the 2009 influenza A (H1N1) pandemic in Norway? Med Hypotheses

Surveillance of respiratory viral infections by rapid immunofluorescence diagnosis, with emphasis on virus interference

Seasonal influenza vaccination and the risk of infection with pandemic influenza: a possible illustration of non-specific temporary immunity following infection

Effect of influenza A virus infection on natural and adaptive cellular immunity

Non-specific immunological effects of selected routine childhood immunisations: systematic review

The case testnegative design for studies of the effectiveness of influenza vaccine

Factors that may impact on immunosenescence: an appraisal

Effectiveness of trivalent flu vaccine in healthy young children

The effectiveness of influenza vaccination in preventing hospitalizations in children in Hong Kong

Association between hospitalization with community-acquired laboratoryconfirmed influenza pneumonia and prior receipt of influenza vaccination

Vaccine effectiveness against laboratory-confirmed influenza in healthy young children: a case-control study

Influenza vaccine effectiveness estimates for Western Australia during a period of vaccine and virus strain stability

Estimates of 2012/13 influenza vaccine effectiveness using the case test-negative control design with different influenza negative control groups

Influenza vaccine effectiveness for hospital and community patients using control groups with and without non-influenza respiratory viruses detected

Influenza vaccine effectiveness in Australia: results from the Australian Sentinel Practices Research Network

Influenza vaccination is not associated with detection of noninfluenza respiratory viruses in seasonal studies of influenza vaccine effectiveness

Potential effect of virus interference on influenza vaccine effectiveness estimates in test-negative designs

Assessment of virus interference in a test-negative study of influenza vaccine effectiveness

Influenza vaccine effectiveness estimates in the Dutch population from 2003 to 2014: the test-negative design case-control study with different control groups

Interval between infections and viral hierarchy are determinants of viral interference following influenza virus infection in a ferret model

Time lines of infection and disease in human influenza: a review of volunteer challenge studies

We thank Julie Au and Vicky Fang for technical assistance.The funders played no role in study design, data collection and analysis, the decision to publish, or preparation of the manuscript.B.J.C. has received research funding from Sanofi Pasteur (Lyon, France).