key: cord-0835086-2wr3b3k8 authors: Lewnard, J. A.; Patel, M. M.; Jewell, N. P.; Verani, J. R.; Kobayashi, M.; Tenforde, M.; Dean, N. E.; Cowling, B. J.; Lopman, B. A. title: Theoretical framework for retrospective studies of the effectiveness of SARS-CoV-2 vaccines date: 2021-01-22 journal: nan DOI: 10.1101/2021.01.21.21250258 sha: bb4077a10bbb5c3ae4bb577f51badb395b7cf303 doc_id: 835086 cord_uid: 2wr3b3k8 Observational studies of the effectiveness of vaccines to prevent COVID-19 are needed to inform real-world use. These are now in planning amid the ongoing rollout of SARS-CoV-2 vaccines globally. While traditional case-control (TCC) and test-negative design (TND) studies feature prominently among strategies used to assess vaccine effectiveness, such studies may encounter important threats to validity. Here we review the theoretical basis for estimation of vaccine direct effects under TCC and TND frameworks, addressing specific natural history parameters of SARS-CoV-2 infection and COVID-19 relevant to these designs. Bias may be introduced by misclassification of cases and controls, particularly when clinical case criteria include common, non-specific indicators of COVID-19. When using diagnostic assays with high analytical sensitivity for SARS-CoV-2 detection, individuals testing positive may be counted as cases even if their symptoms are due to other causes. The TCC may be particularly prone to confounding due to associations of vaccination with healthcare-seeking behavior or risk of infection. The TND reduces but may not eliminate this confounding, for instance if individuals who receive vaccination seek care or testing for less-severe infection. These circumstances indicate the two study designs cannot be applied naively to datasets gathered through public health surveillance or administrative sources. We suggest practical strategies to reduce bias in vaccine effectiveness estimates at the study design and analysis stages. Non-randomized studies undertaken after vaccine authorization or licensure provide crucial information about real-world vaccine effectiveness (VE) . Such studies may also address questions not answered by clinical trials, such as the duration of protection, the effectiveness of alternative dosing schedules, and the level of protection achieved against emerging variants or within various population subgroups. 1, 2 While prospective cohort studies provide an opportunity to compare outcomes among vaccinated and unvaccinated individuals, such studies may be logistically prohibitive due to the large size and complexity of longitudinal follow-up. Thus, retrospective studies comparing prior vaccination among individuals with a known clinical outcome of disease or no disease (i.e. case-control VE studies) provide an efficient alternative. 3 Phase III randomized controlled trials (RCTs) have established efficacy of multiple vaccines against COVID-19, 4-6 with others in progress. Observational VE studies have been prioritized by public health 7,8 and regulatory 9 authorities to inform real-world use. Traditional case-control (TCC) and test-negative design (TND) studies are the primary retrospective frameworks for assessing VE, 10 and plans to use such study designs to assess effectiveness of SARS-CoV-2 vaccines are being developed. 7 However, these and other observational designs may differ with respect to measures of VE and risks of bias. 2 Here we lay out the theoretical basis for use of the TCC and TND in assessments of vaccine direct effects against SARS-CoV-2 infection and COVID-19. We highlight key assumptions and potential biases underlying VE estimates from the TCC and TND studies, and provide practical recommendations to guide researchers through stages of study design, analysis, and interpretation. The TCC and TND derive VE estimates by comparing the odds of prior vaccination among individuals who, at the time of enrollment, are cases (experiencing the clinical endpoint against which VE is to be measured) or non-cases (controls). Cases are typically individuals who meet clinical criteria such as the presence of predefined symptoms together with laboratory-confirmed detection of a vaccine-targeted infectious agent. 11 Under the TCC, controls are selected from a pool of individuals who are not known to be experiencing the clinical endpoint of interest at the time of enrollment, but are members of the same population from which cases are identified. Such individuals may be community controls, selected from among asymptomatic individuals in the community; registry controls, selected from population-based registries; or healthcare controls, selected among patients experiencing an alternate disease unrelated to the pathogen and vaccine of interest. 12 The TND is an alternative to the TCC that has become the default approach for studies of vaccines against influenza, rotavirus, and other pathogens. 13 In the TND, both cases and controls are selected from among individuals who receive diagnostic tests for a pathogen of interest; conventionally, individuals are tested because they experience a clinical syndrome which may be caused by the vaccine-targeted pathogen or other agents. Cases are those testing positive, and controls are those testing negative, placing a heavy reliance on diagnostic accuracy. 2, 14, 15 Since the TND does not require active selection of controls from the community or other settings, it may offers logistical advantages over the TCC. Below we lay out the general framework for estimation of vaccine direct effects in TCC and TND studies. We will introduce relevant notation and assumptions and then extend this framework to address observations that may be expected in studies of SARS-CoV-2. Consider the instantaneous risk (or prevalence) of a clinical condition of interest to be p (in an unvaccinated population), a value between 0 and 1 ( Table 1 ). Consider that vaccination reduces individuals' risk of the condition by a factor VE = (1 − ), such that the relative risk of the condition for vaccinated (versus unvaccinated) individuals is ; we define 1 − as the vaccine "direct effect" resulting from reduction in biological susceptibility to the condition of interest due to vaccination. Vaccine protection could be achieved through differing means, such as uniformly reducing individuals' risk by a factor of (1 − ), or by preventing the condition with certainty among a proportion (1 − ) of recipients while failing to confer any protection among others. Unless otherwise noted in our summary, these differing mechanisms of vaccine action (elsewhere termed "leaky" or "all-or-nothing" protection 10 , respectively) do not result in practical differences in our analyses of VE. Defining as the proportion of the population receiving vaccine, the probabilities of individuals in the population falling into various case and control categories, given their vaccination status, is laid out in the two-by-two tables in Table 2 . Under the TCC, which, for rare outcomes ( ≈ 0), yields an unbiased estimate VE * !"" ≈ 1 − = VE. Alternative design approaches such as incidence density matching of cases and controls, 16 inclusive sampling, 17 or other strategies may reduce reliance on the rare outcome assumption; here for simplicity we focus on more general TCCs designs. Under the TND, controls instead experience an outcome that is unaffected by vaccination against the pathogen of interest. 14, 15, 18 Defining the risk of this outcome as k, regardless of vaccination ( Table 2) , which does not require the rare-disease assumption to recover the vaccine direct effect. Vaccine-conferred protection may impact multiple aspects of the natural history of infectious disease agents such as SARS-CoV-2, including but not limited to prevention of virus acquisition, reduced viral replication in the upper respiratory mucosa, earlier clearance of infection, and prevention of mild or severe symptoms. The differential effects of SARS-CoV-2 vaccines against infection and disease endpoints remain imperfectly understood 19 and may vary by vaccine product and class. The typical estimands of interest in TCC and TND studies are VE against symptomatic disease of any severity, or VE against severe disease. These are also typical target estimands in individually-randomized vaccine efficacy trials. 20 Consider % to be the relative risk of infection for a vaccinated versus unvaccinated individual, which may result from reductions in host susceptibility to acquisition of SARS-CoV-2 or accelerated clearance of the pathogen. 20 Consider & to be the relative risk of COVID-19 (or COVID-19 meeting a particular severity threshold, e.g. hospitalization, intensive care unit admission, or mechanical ventilation) for an infected vaccinated individual versus an infected unvaccinated individual. Protection resulting from prevention of both infection and progression is then VE ' = 1 − % & ; we assume VE ' is the desired estimand and describe biases of VE * !"" and VE * !#$ relative to VE ' . Specialized designs would be required to isolate % or & . Prolonged viral shedding and the presence of asymptomatic infection may introduce misclassification in both TCC and TND studies. For instance, TCC studies enrolling controls selected from the community, from registries, or among patients seeking care for other (e.g., non-respiratory) diseases, may suffer from misclassification if a proportion of these controls are in fact infected with SARS-CoV-2, but not tested due to the absence of symptoms. Inclusion of infected individuals in the control group will diminish the apparent effects of vaccination, provided vaccine prevents infection. Likewise, in both TCC and TND studies, individuals who are shedding SARS-CoV-2 genetic material but experiencing illness due to another cause (e.g., another infection or a non-infectious process with overlapping symptoms such as chronic obstructive pulmonary disease) may be misclassified as cases. Misclassification of asymptomatically infected persons as controls, and of infected symptomatic persons as cases when their illness is not caused by SARS-CoV-2, can be expected to increase with higher prevalence of infection in the community. To formalize resulting biases, consider separately the risks of SARS-CoV-2 acquisition and disease progression. Defining a as the instantaneous risk (prevalence) of being infected with SARS-CoV-2, at any time, and d as the instantaneous risk of experiencing symptoms, given infection, the risk of symptomatic and asymptomatic SARS-CoV-2 infections are and (1 − ), respectively. We extend our contingency tables to accommodate test-positive symptomatic cases, asymptomatic controls, and test-negative symptomatic controls in Table 3 . Here, the TCC yields which remains subject to bias owing to the risk of symptoms due to causes other than SARS-CoV-2 (k) and the prevalence of SARS-CoV-2 infection (a). However, these biases may be ignorable under several conditions. If prevalence of other conditions is low or negligible ( ≈ 0, as may be true for very severe endpoints), or negligible relative to the risk of COVID-19 for an infected individual ( ≪ , if endpoints are chosen with high specificity for COVID-19), the TCC yields Such a design may under-estimate VE ' if prevalence of SARS-CoV-2 infection (a) is high owing to inclusion of infected asymptomatic individuals in the control group. If prevalence of SARS-CoV-2 infection in the community is low or negligible ( ≈ 0), VE * !"" ≈ 1 − % & . Comparing symptomatic individuals who test positive and negative for infection, the TND yields The same conditions by which bias can be reduced under the case-control design reduce bias under the test-negative design. With ≈ 0 or ≪ , We illustrate the quantitative extent of bias under the TCC and TND in Figure 1 . Under most realistic situations, including a highly efficacious vaccine (VE '~9 5%), low prevalence of acute infection ( < 0.05), and substantially increased risk of symptoms, given infection ( ≫ ), bias is low or negligible for both designs. While relatively insensitive to changes in prevalence of infection within ranges that have been reported for SARS-CoV-2, bias increases if the risk of disease, given infection, is low (i.e. if non-specific All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted January 22, 2021. ; or prevalent symptoms are included in clinical case criteria), potentially leading to substantial underestimation of VE ' . Multiple assays are available for SARS-CoV-2 detection, and the use of assays with differing performance characteristics may influence the extent of biases identified above. Nucleic acid amplification assays such as polymerase chain reaction (PCR), for instance, have higher analytic sensitivity (i.e., likelihood of positive test for sample containing viral material at the minimum detectable concentration for which assay is designed), potentially leading to detection of SARS-CoV-2 genetic material among individuals with low levels of virus shedding. Antigen detection tests, with lower analytic sensitivity, may in contrast yield negative results for individuals with low virus shedding that is detectable by molecular testing. 26 In comparison to mild illness, both assays may have lower sensitivity for severe disease which can be delayed in presentation, occurring often during the second week after infection onset when viral shedding is lower. 42 Define & and % as the sensitivity of an assay for detecting SARS-CoV-2 among individuals experiencing COVID-19 and those not experiencing symptoms attributable to SARS-CoV-2, respectively. We limit our consideration to differences in test sensitivity, considering specificity to encompass the related issue of non-etiologic SARS-CoV-2 detections; under field conditions, antigen and molecular tests have been found to have similar analytical specificity. 26, 27 The limiting case of an assay which perfectly rules out SARS-CoV-2 detections that are not causing symptoms ( % = 0) yields the contingency table presented in Table 4 for case and control groups (Table S1 extends this for non-zero values of % ). With % = 0, the TCC yields removing the reliance on assumptions of ≈ 0 or ≪ , and allowing VE * !"" ≈ 1 − % & = VE $ if overall prevalence of infection is low ( ≈ 0). Under the TND, Here, we obtain an unbiased estimate VE * We note that no available assay enables perfect differentiation of COVID-19 from non-etiologic detections of SARS-CoV-2; statistical corrections for imperfect test sensitivity and specificity have been proposed to adjust final VE estimates. 28 Further complicating the selection of assays, available data suggest similar viral loads and/or nucleic acid abundance among individuals with or without symptoms at similar stages of infection. 29,30 However, onset of COVID-19 upper respiratory tract symptoms typically coincides with peak viral shedding. Thus, individuals seeking care for new-onset COVID-19 may generally have higher viral load than individuals experiencing symptoms owing to other causes, who may be at later stages of SARS-CoV-2 shedding or molecular detection. 31 This circumstance may drive an artefactual association of higher SARS-CoV-2 shedding with COVID-19. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250258 doi: medRxiv preprint Differential healthcare-seeking behaviors of individuals who can access and receive vaccines, versus others, may be another source of bias. The TND aims to limit such bias by restricting enrollment to individuals who seek care and receive diagnostic testing for the same or similar clinical indications. In TCC studies, there may be uncertainty about whether controls were truly disease-free, as some may not have sought care given symptoms. This may be particularly true if studies select controls from registries without direct outreach. We may quantify the resulting bias by considering the number of controls who are misclassified when the probability of seeking care, given symptoms, is ( among the vaccinated and ) among the unvaccinated. Defining eligible controls as individuals who do not become ascertained as cases ( Table 5) : The resulting bias generally leads to under-estimation of true protection (VE $ ) when the probability of seeking treatment is higher among vaccinated than unvaccinated persons, given the same status of being infected or symptomatic, owing to the disproportionately higher likelihood for vaccinated individuals to be ascertained as cases (Figure 3) . However, studies enrolling health facility controls may encounter further difficulty. If vaccination is associated with higher rates of healthcare seeking, prevalence of vaccination among controls seeking care for less-severe conditions may be inflated, leading to overestimation of VE $ . Applying the same formulation under the TND, the terms ( and ) cancel out (Table 5) . Nonetheless, in practical contexts, TND studies may enroll individuals across a spectrum of clinical severity. This may cause bias to persist if the probability of treatment-seeking differs for vaccinated and unvaccinated individuals with relatively less severe or more severe illness. 18 These considerations motivate approaches such as matching of cases and test-negative controls (or cases and alternative-disease controls under the TCC) on clinical severity. Prevalence of other causes of illness leading to care-seeking (e.g., among alternative-disease controls under the TCC and among test-negative controls under the TND) may vary considerably over time. If risk of disease due to these other causes is also associated with likelihood of vaccination, further bias may arise. Even under normal circumstances, disease etiology among controls can change within a season or from year to year. This variation is likely to be extreme over the time horizon that SARS-CoV-2 vaccines are rolled out, as non-pharmaceutical interventions (NPIs) targeting SARS-CoV-2 have affected circulation of other pathogens. 32,33 As NPIs are rolled back, other causes of healthcare-seeking may surge. Since many infections causing respiratory symptoms (e.g., influenza, pneumococcus) are vaccine preventable, controls with these infections may be less likely to have sought vaccine. Accordingly, the ratio ( / ) may change over time. Testing for SARS-CoV-2 is commonly undertaken for both clinical diagnostic and public health purposes. Thus, data on SARS-CoV-2 test results may encompass individuals tested because of clinical symptoms as well as those tested due to contact with a known or suspected COVID-19 case, for purposes of screening before work or medical procedures, or for other indications. Stratifying analyses to address fairly homogeneous populations, or to individuals who receive SARS-CoV-2 testing with the same frequency or for the same indications, may be of considerable importance. Variation in the likelihood of seeking testing is expected to be greatest for cases of mild illness or in the absence of any symptoms. There may be substantial association with individuals' healthcare utilization, including the propensity to be vaccinated. In the most extreme case, populations targeted for vaccination may also be populations targeted for frequent testing, such as healthcare providers, residents of longterm care facilities, and essential workers. Infections in these groups will be more likely to be detected relative to infections within groups without similar access to testing, who may also be less likely to be vaccinated. Here ( ≫ ) , potentially leading to substantial under-estimation of VE. Similar bias may persist when vaccines are made more widely available if individuals with the greatest propensity to seek healthcare, given the same clinical presentation, including SARS-CoV-2 testing, are also more likely to be vaccinated. Thus, the TND cannot be applied naively to comprehensive SARS-CoV-2 testing datasets gathered through public health surveillance or administrative data sources. To adequately correct for potential differences in vaccinated and unvaccinated individuals' likelihood of being tested, designs or analyses should compare individuals being tested for the same indications (e.g., fulfilling particular clinical criteria, known high-risk exposure to SARS-CoV-2, or routine screening), for instance by matching or stratifying on occupation. Bias may be further reduced in studies considering severe disease endpoints for which both vaccinated and unvaccinated individuals would be unlikely to defer treatment, regardless of their propensity to seek healthcare for less severe illness. Limited vaccine supplies are being targeted at groups most likely to be exposed to SARS-CoV-2 as well as those at greatest risk of severe COVID-19, if infected. Even when vaccines are more widely available, uptake of vaccination may be highest among individuals who adhered most strongly to NPIs. Accordingly, exposure to infection and risk of progressing to severe disease will not be equal between vaccinated and unvaccinated groups. This differential extends not only to rates of new infection, but also to the prevalence of prior natural infection and immunity. As available evidence suggests naturally acquired immunity is strongly (~90%) protective against re-infection, there may be substantial differences in risk among vaccinated and unvaccinated individuals owing to factors other than vaccine-derived protection. 34, 35 We extend the contingency tables laid out previously (Table 6 ) to incorporate , the relative risk of SARS-CoV-2 infection among vaccinated (vs. unvaccinated) individuals due to factors other than vaccination, as well as ( and ) , the proportions of the vaccinated and unvaccinated populations, respectively, that remain uninfected. For simplicity, we assume near-total protection against reinfection conferred by naturally acquired immunity. Under the two designs, which, with low prevalence of infection ( ≈ 0) and low risk of disease due to other causes ( ≈ 0 or ≪ ), yield Here, differential risk of SARS-CoV-2 infection and prevalence of immunity among vaccinated and unvaccinated individuals lead to counteracting biases. With higher risk among individuals who receive vaccination ( > 1), we may expect higher prevalence of naturally acquired infection in this group and ( < ) ; bias is offset fully if = ) / ( . We plot conditions under which bias is driven predominantly by differential exposure to infection or by differential naturally acquired immunity in Figure 4 . Considering the context of VE studies undertaken shortly after vaccine implementation, we may assume individuals' time at risk is contributed primarily in All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted January 22, 2021. ; the period before vaccination; with exponentially distributed infection times, and a cumulative force of infection exerted over the period at risk, we may define the remaining uninfected populations ) = exp (− ) and ( = exp (− ). Bias is fully offset with = log( ) /( − 1), yielding ) = ! !"# (Figure 4) . Thus, the prevalence of pre-existing naturally acquired immunity needed to offset bias resulting from differential exposure is high, at 33% and 56% for = 5 and = 1.5, respectively. Detailed risk factor data are likely necessary to address confounding associated with differential risk of infection among individuals receiving and those not receiving vaccination. While certain occupations (e.g. healthcare worker or other essential worker) may be associated with heightened risk of exposure relative to the general public, substantial variation in risk may exist within these categories owing to differences in the specific tasks conducted and availability of as well as adherence to infection control measures. Individuals may modify aspects of their behavior following vaccine receipt, further altering differences in risk of infection among vaccine recipients and non-recipients. Collection of questionnaire items addressing risk behaviors and exposures is thus of importance to mitigate confounding. If it is possible to control for differences in individuals' risk of infection, vaccine-unrelated differences in rates of infection and prevalence of natural immunity are not a prominent source of bias in the short term ( = 1 and ) = ( ). While we focus above on studies undertaken shortly after vaccine rollout, prior work has considered differential accrual of natural immunity among vaccinated and unvaccinated individuals over longer periods of time. 3,18,36,37 For vaccines conferring "leaky" protection, TCC and TND studies may underestimate true protection owing to higher rates of acquisition of naturally-acquired immunity among unvaccinated individuals. This bias does not arise, however, for vaccines conferring "all-or-nothing" protection. 3,18 Our analysis provides several practical insights to inform upcoming TCC and TND studies of SARS-CoV-2 vaccines. While we identify conditions and reasonable assumptions under which both designs can supply reliable VE estimates, several threats to validity should be taken into account in the design, analysis, and interpretation of TCC and TND studies. Below we offer considerations for researchers undertaking TCC and TND studies of vaccines against SARS-CoV-2: Use of severe COVID-19 clinical endpoints: Use of severe disease endpoints minimizes risk of misclassification and healthcare-seeking bias. To allow sufficient power, studies may need to enroll patients across multiple healthcare facilities, underscoring the need for standardized clinical criteria and case definitions. Studies addressing VE against common or less-severe clinical endpoints may benefit from using extended analytic methods that have been proposed previously for estimation of VE against diseases with non-specific diagnostic criteria. 11,38,39 As these analysis methods must generally account for the strength of association between SARS-CoV-2 infection and clinical symptoms, enrollment and testing of control groups without symptoms related to SARS-CoV-2 infection may be of value for researchers studying VE against such endpoints. Use of molecular or antigen tests: Studies using SARS-CoV-2 detection assays with the highest analytic sensitivity may encounter the greatest risk of misclassifying symptomatic individuals at the end of their course of infection as cases, including when SARS-CoV-2 is not the cause of symptoms. This could be more problematic when prevalence of SARS-CoV-2 infection is high and outcomes of interest include non-severe or non-specific COVID-19 clinical manifestations, such as in outpatient settings. However, trade-offs associated with risk of bias must be balanced against assay specificity and test performance under varied prevalence and clinical settings. Where molecular assays with high analytical sensitivity are preferred for clinical care provision, researchers may consider analyzing quantitative cycle threshold data or collecting a second specimen for antigen-based testing. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Reliance on clinic-based testing under both designs makes it difficult to identify and correct for differences in the likelihood for vaccinated and unvaccinated individuals, given infection and symptoms, to be ascertained as cases. Collection of standardized data on clinical symptoms among both cases and controls may reduce the risk of bias related to vaccine-associated differences in healthcare access or healthcare seeking. This strategy further provides an opportunity to identify and exclude, if applicable, registry-based or community controls who were ill but did not seek care. If testing is conducted for multiple pathogens, researchers may also investigate bias by assessing VE against other causes of illness not prevented by SARS-CoV-2 vaccination as a "sham" analysis to indicate risk of bias. 40, 4140, 41 Background immunity: Prevalent naturally acquired immunity may complicate the interpretation of results under both the TCC and TND, potentially contributing to lower risk of infection and disease among individuals with the greatest risk of SARS-CoV-2 exposure. As initial vaccine doses will be prioritized to groups with high infection risk, individuals eligible to receive vaccine at the earliest opportunity may have higher prevalence of pre-existing naturally acquired immunity than those in lower-priority tiers. Studies may mitigate this bias, in part, by comparing individuals within the same prioritization groups, by collecting detailed risk factor data to control for variation in exposure, and through use of serological assays (if available) that differentiate vaccine-derived or naturally-induced responses (e.g. to non-vaccine antigens). Lewnard JA. Uses of pathogen detection data to estimate vaccine direct effects in case-control All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Risk ratio for a clinical outcome of interest among counterfactually vaccinated versus unvaccinated individuals; we define ! as the risk ratio of shedding the pathogen (owing to differential acquisition among the vaccinated) and " as the risk ratio of disease progression, given infection. Vaccine effectiveness is defined as VE = 1 − in the general case. The combined extent of protection resulting from prevention of shedding and progression is VE # = 1 − ! " . Test sensitivity, or the probability of SARS-CoV-2 detection in an infected patient. We further define ! as sensitivity for detecting SARS-CoV-2 when the pathogen is not causing symptoms, and " as sensitivity for detecting SARS-CoV-2 when the pathogen is the cause of symptoms. 0 to 1 Probability that individuals who experience the clinical outcome of interest will receive a test. We define $ and % for vaccinated and unvaccinated individuals, respectively, to account for potential differences in healthcare access or healthcare seeking associated with vaccination. 0 to 1 Relative risk of SARS-CoV-2 infection among vaccinated (versus unvaccinated) individuals, owing the factors other than vaccination (e.g., differential exposure or susceptibility to infection). > 1 corresponds to a scenario of higher risk among those who will receive vaccination; 0 ≤ < 1 corresponds to a scenario of lower risk among the vaccinated. Proportion of the population that remains uninfected. We define $ and % for vaccinated and unvaccinated individuals, respectively, to account for potential differences in risk of infection within these populations, apart from vaccineconferred immunity. For the case of exponentially distributed infection times, we consider % = exp (− ) and $ = exp (− ). All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250258 doi: medRxiv preprint , proportion of the population vaccinated; , risk of the outcome of interest; , risk of the negative-control outcome; , relative risk of the outcome of interest, given vaccination ( Table 1) . Outcome Test-positive case (symptomatic) Community control (asymptomatic) Test-negative control (symptomatic) the table are probabilities of the various listed outcomes, accounting for vaccinated or unvaccinated status. Parameters: , proportion of the population vaccinated; , prevalence of SARS-CoV-2 infection; , risk of COVID-19 among those with SARS-CoV-2 infection; , risk of symptoms due to causes other than SARS-CoV-2 infection; ! , relative risk of SARS-CoV-2, given vaccination; " , relative risk of COVID-19, among those with SARS-CoV-2 infection, given vaccination ( Table 1) . Exposure Outcome Test-positive case (symptomatic) Community control (asymptomatic) Test-negative control (symptomatic) Vaccinated Contents of the table are probabilities of the various listed outcomes, accounting for vaccinated or unvaccinated status. Parameters: , proportion of the population vaccinated; , prevalence of SARS-CoV-2 infection; , risk of COVID-19 among those with SARS-CoV-2 infection; , risk of symptoms due to causes other than SARS-CoV-2 infection; ! , relative risk of SARS-CoV-2, given vaccination; " , relative risk of COVID-19, among those with SARS-CoV-2 infection, given vaccination; " , sensitivity for SARS-CoV-2 detection among individuals experiencing COVID-19 ( Table 1) . We present a more general contingency table for non-zero values of ! (test sensitivity for SARS-CoV-2 infection among those not experiencing COVID- 19) in Table S1 . (1 − )(1 − ) % Contents of the table are probabilities of the various listed outcomes, accounting for vaccinated or unvaccinated status. Parameters: , proportion of the population vaccinated; , prevalence of SARS-CoV-2 infection; , risk of COVID-19 among those with SARS-CoV-2 infection; , risk of symptoms due to causes other than SARS-CoV-2 infection; ! , relative risk of SARS-CoV-2, given vaccination; " , relative risk of COVID-19, among those with SARS-CoV-2 infection, given vaccination; $ , probability of testing for symptomatic vaccinated individuals; % , probability of testing for symptomatic unvaccinated individuals ( Table 1) . All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250258 doi: medRxiv preprint Table 6: Contingency table for TCC and TND studies with differential risk of current and prior infection among the vaccinated and unvaccinated. Outcome Test-positive case (symptomatic) Community control (asymptomatic) Test-negative control (symptomatic) Vaccinated Contents of the table are probabilities of the various listed outcomes, accounting for vaccinated or unvaccinated status. Parameters: , proportion of the population vaccinated; , prevalence of SARS-CoV-2 infection; , risk of COVID-19 among those with SARS-CoV-2 infection; , risk of symptoms due to causes other than SARS-CoV-2 infection; ! , relative risk of SARS-CoV-2, given vaccination; " , relative risk of COVID-19, among those with SARS-CoV-2 infection, given vaccination; , relative risk of exposure or infection among vaccinated versus unvaccinated individuals attributable to factors other than vaccine-conferred immunity; $ , proportion of vaccinated individuals who remain uninfected; % , proportion of unvaccinated individuals who remain uninfected ( Table 1) . All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted January 22, 2021. ; (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted January 22, 2021. ; Table 1 . Red diagonal lines across each panel illustrate the true effect. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted January 22, 2021. ; Table 1 ; solid and dashed lines correspond to low ( = 0.01) and high ( = 0.1) infection prevalence scenarios, respectively. Red diagonal lines across each panel illustrate the true effect. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250258 doi: medRxiv preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250258 doi: medRxiv preprint All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250258 doi: medRxiv preprint Vaccine effects and impact of vaccination programmes in post-licensure studies Observational studies and the difficult quest for causality: Lessons from vaccine effectiveness and impact studies Selection of controls in case-control studies: II. Types of controls The use of test-negative controls to monitor vaccine effectiveness The test-negative design for estimating influenza vaccine effectiveness The case test-negative design for studies of the effectiveness of influenza vaccine On the need for the rare disease assumption in case-control studies Use of the case-control approach in vaccine evaluation: efficacy and adverse effects Measurement of vaccine direct effects under the test-negative design Understanding COVID-19 vaccine efficacy Study designs for evaluating different efficacy and effectiveness aspects of vaccines Efficacy of a bivalent L1 virus-like particle vaccine in prevention of infection with human papillomavirus types 16 and 18 in young women: a randomised controlled trial Efficacy of the RTS,S/AS02A vaccine against Plasmodium falciparum infection and disease in young African children: randomised controlled trial First human challenge testing of a pneumococcal vaccine: Double-blind randomized controlled trial SARS-CoV-2, SARS-CoV, and MERS-CoV viral load dynamics, duration of viral shedding, and infectiousness: a systematic review and meta-analysis Occurrence and transmission potential of asymptomatic and presymptomatic SARSCoV-2 infections: A living systematic review and metaanalysis Field performance and public health response using the BinaxNOW TM Rapid SARS-CoV-2 antigen detection assay during community-based testing Field evaluation of a rapid antigen test (Panbio TM COVID-19 Ag Rapid Test Device) for COVID-19 diagnosis in primary healthcare centres Influenza in the COVID-19 Era Antibody status and incidence of SARS-CoV-2 infection in health care workers Neutralizing antibodies correlate with protection from SARS-CoV-2 in humans during a fishery vessel outbreak with a high attack rate The measurement and interpretation of age-specific vaccine efficacy Waxing Understanding of Waning Immunity Bias due to differential and non-differential disease-And exposure misclassification in studies of vaccine effectiveness Estimating malaria vaccine efficacy in the absence of a gold standard case definition: Mendelian factorial design Negative Controls: A tool for detecting confounding and bias in observational studies Childhood vaccines and antibiotic use in low and middle income countries Population point prevalence of SARS-CoV-2 infection based on a statewide random sample -Indiana REACT-1 round 6 updated report high prevalence of SARS-CoV-2 swab positivity with reduced rate of growth in England at the start of Community transmission of severe acute respiratory syndrome coronavirus 2 disproportionately affects the Latinx population during shelter-in-place in San Francisco Estimating the burden of SARS-CoV-2 in France Incidence, clinical outcomes, and transmission dynamics of severe coronavirus disease 2019 in California and Washington, United States: prospective cohort study