key: cord-292372-kn27ghlv
authors: de Chaisemartin, Clément; de Chaisemartin, Luc
title: BCG vaccination in infancy does not protect against COVID-19. Evidence from a natural experiment in Sweden
date: 2020-08-23
journal: Clin Infect Dis
DOI: 10.1093/cid/ciaa1223
sha: 
doc_id: 292372
cord_uid: kn27ghlv

BACKGROUND: The Bacille Calmette-Guérin (BCG) tuberculosis vaccine has immunity benefits against respiratory infections. Accordingly, it has been hypothesized to have a protective effect against COVID-19. Recent research found that countries with universal BCG childhood vaccination policies tend to be less affected by the COVID-19 pandemic. However, such ecological studies are biased by numerous confounders. Instead, this paper takes advantage of a rare nationwide natural experiment that took place in Sweden in 1975, where discontinuation of newborns BCG vaccination led to a dramatic fall of the BCG coverage rate, thus allowing us to estimate the BCG’s effect without the biases associated with cross-country comparisons. METHODS: Numbers of COVID-19 cases and hospitalizations were recorded for birth cohorts born just before and just after 1975, representing 1,026,304 and 1,018,544 individuals, respectively. We used regression discontinuity to assess the effect of BCG vaccination on Covid-19 related outcomes. This method used on such a large population allows for a high precision that would be hard to achieve using a randomized controlled trial. RESULTS: The odds ratio for Covid-19 cases and Covid-19 related hospitalizations were 1·0005 (CI95: [0·8130-1·1881]) and 1·2046 (CI95: [0·7532-1·6560]), allowing us to reject fairly modest effects of universal BCG vaccination. We can reject with 95% confidence that universal BCG vaccination reduces the number of cases by 19% and the number of hospitalizations by 25%. CONCLUSIONS: While the effect of a recent vaccination must be evaluated, we provide strong evidence that receiving the BCG vaccine at birth does not have a protective effect against COVID-19 among middle-aged individuals.

The Bacille Calmette-Guérin (BCG) vaccine is an attenuated live vaccine that has been proven effective against tuberculosis, in particular its severe manifestations like meningeal and miliary tuberculosis [1] . Besides its specific effects, the vaccine has immunity benefits against non-targeted pathogens [2] , and in particular against respiratory infections caused by RNA viruses like influenza [3] .

These effects are thought to be mediated mostly by "trained immunity", a recently described mechanism of epigenetic reprogramming of innate immune cells [4] . While this mechanism is still under investigation, most studies show that these effects tend to wane after 15 to 20 years [5, 6] .

Since SARS-Cov-2 is also a single-stranded RNA virus, it has been hypothesized that differences in BCG vaccination coverage could explain the wide differences in disease burden observed between countries. A pioneering preprint paper by Miller et al. found that countries with universal Bacillus Calmette-Guérin (BCG) childhood vaccination policies tend to be less affected by the COVID-19 pandemic, in terms of their number of cases and deaths [7] . While unpublished, this study had a great impact and gave rise to many comments and follow-up studies (reviewed in [8] ). Some published studies were able to replicate this result [9] [10] [11] , but several authors underlined the important statistical flaws inherent to such ecological studies and concluded that randomized controlled trials (RCT) were necessary to address the question [8, 12] . As of June 5th 2020, no less than 12 randomized controlled trials (RCTs) studying the protective effect of the BCG against COVID-19 are already registered on https://clinicaltrials.gov/. However, none has a primary completion date earlier than October 1 st 2020, so these RCTs' first results will not be known until at least five or six months. With the epidemic still on the rise worldwide, and in the absence of a SARS-Cov-2 vaccine, there is an urgent need to know if BCG non-specific effects could be harnessed as a substitute prophylactic treatment.

Regression discontinuity (RD) is a method designed by social scientists to assess the effect of an exposure on an outcome. It is deemed as reliable as RCTs to tease out causality from correlation [13] , A c c e p t e d M a n u s c r i p t 6 and typically yields results similar to those obtained in RCTs [14, 15] . In this paper, we applied this method to a rare natural experiment that took place in Sweden. Sweden currently has the 5th highest ratio of COVID-19 deaths per capita in the world. In April 1975, it stopped its newborns BCG vaccination program, leading to a dramatic drop of the BCG vaccination rate from 92% to 2% for cohorts born just before and just after the change [16] . We compared the number of COVID-19 cases and hospitalizations per capita, for cohorts born just before and just after April 1975, representing 1,026,304 and 1,018,544 individuals, respectively. Using RD, we were able to show that those cohorts do not have different numbers of COVID-19 cases and hospitalizations per capita, with a high precision that would hardly be possible to reach with a RCT design. 

Regression discontinuity (RD) is a commonly-used method to measure the effect of a treatment on an outcome [18] . It is applicable when only individuals that satisfy a strict criterion are eligible for a policy. Then, RD amounts to comparing the outcome of interest among individuals just above and just below the eligibility threshold. In this study, RD will amount to comparing the number of COVID-19 cases, hospitalizations, and deaths among individuals born just before and just after April 1st

1975. The effect of universal BCG vaccination for individuals born around April 1st, 1975 was estimated using the state-of-the-art estimator for RD [19] . The estimator amounts to comparing treated and control units, in a narrow window around April 1st 1975. It uses linear regressions of the outcome on birth cohort to the left and to the right of the threshold, to predict the outcome of treated and untreated units at the threshold. Then, the estimator is the difference between these two predicted values. The estimator and 95% confidence interval were computed using the rdrobust Stata command, see [20] . The RD estimator in [19] assumes that the variable determining exposure is continuous, but it has also been used when that variable is discrete but takes a large number of values. Our analysis is at the quarter-of-birth level, so the variable determining exposure to BCG vaccination is not continuous, but it takes a large number of values. For instance, we observe Cases/1000 inhabitants for 288 QBCs.

The validity of the RD estimator is usually assessed by testing if observations to the left and to the right of the threshold have similar characteristics [13] . In the Supplementary Table 3 A c c e p t e d M a n u s c r i p t 8 Finally, because we observe deaths for groups of 3 consecutive YBCs till 1980, there are only two data points after 1975 for that variable (1975-1977 and 1978-1980) , so the RD method cannot be used. In the Supplemental Materials, we merely use a t-test to compare the death rates in the 1972-1974 and 1975-1977 YBCs. This method is less robust than the RD method we use for cases and hospitalizations, as it does not account for the fact that the average age of the treated and control groups differs by 3 years.

This study uses the number of COVID-19 cases per 1000 inhabitants for quarterly birth cohorts born Table 1 for details.

In an RD design, the presence or absence of a treatment effect can be assessed visually, by drawing a scatter-plot with the variable determining eligibility on the x-axis, and the outcome variable on the y-axis. If one observes that the relationship between these two variables jumps discontinuously at the eligibility threshold, this is indicative of a treatment effect. Accordingly, Figure 1 shows no discontinuity in the numbers of COVID-19 cases per 1000 inhabitants for cohorts born just before and just after April 1975. This suggests that universal BCG vaccination has no effect on the number The effects in Table 1 are intention-to-treat effects [21] , as the vaccination coverage pre-1975 was around 92%, and after the 1975 change in policy, about 2% of the population was still vaccinated.

Taking into account foreign-born residents (27·2% of the Swedish population born in 1975), which were not affected by the policy, the policy led to a drop of the vaccination rate of 65.5%. To convert the intention-to-treat effects in Table 1 into the effect of being vaccinated at birth, one needs to divide the intention-to-treat effects and their confidence intervals by 0·655 [22] . In odds ratios, the 

In this study, we took advantage of a change in vaccination policy in Sweden to investigate the link between BCG vaccination in infancy and Covid-19 cases, hospitalizations and deaths, using a regression discontinuity approach.

Contrarily to most studies on the question, we compared Covid-19 cases between two very similar groups of people from the same country. This allows us to get rid of all confounders linked to crosscountries comparisons, and of confounders like sex or socio-economics status that are often present in observational studies that do not rely on a quasi-experimental design, unlike ours. Another strength of this study is its statistical precision. Since we could gather nationwide data in a country

where Covid-19 rates are high, we are able to reject fairly small effects of the BCG vaccine. Achieving a comparable statistical precision in an RCT would require an unrealistically large sample. Even with a COVID-19 hospitalization rate of 0·5%, as among the elderly Swedish population, a randomized controlled trial that could reject BCG effects larger than 25% of the baseline hospitalization rate, as in our study, would require including around 15,000 participants.

While previous studies mostly addressed differences in BCG vaccination policies but did not account for differences in actual BCG coverage, we work with two populations with well documented and very different vaccination rates. The termination of the universal BCG vaccination program in Sweden had dramatic effects on the BCG coverage rate. Based on nationwide reports on the vaccination status of children below 7 sent to the National Bacteriological Laboratory in 1981, 92% of children born in 1974 got vaccinated, against 26% of those born in 1975, and less than 2% of those born between 1976 and 1980 [16] . Among children born in 1975, most of the vaccinated children were born in the first quarter of the year, when the universal vaccination policy was still in place [16] . Prior to that, Sweden had already eliminated its revaccination program at 7 years of age in 1965. Sweden also stopped its revaccination program at 15 years of age in 1986, four years before the 1975 cohort turned 15 years old. Finally, Sweden stopped its vaccination program for conscripts M a n u s c r i p t 11 in 1979, long before the 1975 cohort would do its military service [23] . Overall, children born before April 1st 1975 benefited from a BCG vaccination policy at birth, while children born after that did not benefit from any BCG vaccination policy.

This being said, there is a number of limitations to this work that one has to bear in mind. As in many other countries, Folkhälsomyndigheten's cases count probably underestimates the true number of cases, because it only includes cases confirmed by a laboratory test. Its deaths count probably underestimates the true number of COVID-19 deaths as well, as it only includes deaths where a COVID-19 diagnosis has been confirmed during the past 30 days. However, these limitations are common to all studies on this question and are unlikely to affect much the results. Moreover, cases confirmed by a laboratory test are arguably the more severe ones, given that our data covers a time period where testing resources were scarce in Sweden, as in many other countries.

Our computation of the change of the vaccination rate induced by the 1975 policy relies on the assumption that the BCG vaccination rate of foreign-born residents is the same just before and just after April 1975. However, this is a reasonable assumption, since no other European country changed its BCG vaccination policy in 1975.

Furthermore, RD estimates only apply to units close to the eligibility threshold. For instance, this study estimates the effect of universal BCG vaccination for individuals born around April 1st 1975, who are in their mid-forties during the COVID-19 pandemic, and cannot be generalized to the entire population. It seems reasonable, however, to speculate that our findings hold true for older people since most studies assessing the long-term heterologous effects of the BCG show they tend to fade rather than increase over time [5, 6] . Two studies argue that BCG vaccination at birth could have larger effects against COVID-19 on older individuals, those most at risk during the pandemic, than on middle-aged individuals [7, 11] . Based on the literature, we estimate that this is very unlikely to be the case.

A c c e p t e d M a n u s c r i p t 12 Moreover, this study does not measure the COVID-19 immunity benefit conferred by a recent BCG vaccination, as individuals born just before Q2-1975 were vaccinated 45 years ago. The RCTs currently underway will tell if the protective effect of a recent BCG vaccination differs from the effect measured in this study.

Overall, this study shows BCG vaccination at birth does not have a strong protective effect against COVID-19, at least in middle-aged individuals. Thus, it seems that BCG childhood vaccination policies cannot account for the differences in the severity of the pandemic across countries, as had been hypothesized by prior studies [7, 9, 10] . This advocates for a strict adherence to WHO's recommendation of the vaccine to infants outside of clinical trials [25] , and for restraint from starting new clinical trials on this question. The former point is of particular importance for a vaccine whose lengthy production process regularly leads to worldwide shortages with dire consequences on children from country with high prevalence of tuberculosis [26] . While RCTs will complement this study by measuring the effect of a recent vaccination, this study comes much before results of the RCTs will be made available, and with a greater precision. It exemplifies the potential of leveraging past medical policies and statistical techniques designed in the social sciences to answer current medical questions. (2), and its 95% confidence interval is shown in Column (3). The effects and their 95% confidence interval were computed using the rdrobust Stata command, using linear polynomials on both sides of the threshold, and the robust confidence intervals reported by the command. The rdrobust command first selects observations within a narrow bandwidth around the threshold, using the bandwidth proposed in [19] . Then, linear regressions of the outcome on birth cohort are run to the left and to the right of the threshold, to predict the outcome of treated and untreated units at the threshold.

Finally, the estimator is the difference between these two predicted values. The number of quarters of birth used in the last two steps of the estimation is shown in Column (4). 

Protection by BCG vaccine against tuberculosis: a systematic review of randomized controlled trials

The non-specific and sex-differential effects of vaccines

Non-specific effects of BCG vaccine on viral infections

Targeting innate immunity for tuberculosis vaccination

The duration of protection of school-aged BCG vaccination in England: a population-based case-control study

Duration of BCG protection against tuberculosis and change in effectiveness with time since vaccination in Norway: a retrospective population-based cohort study

Correlation between universal BCG vaccination policy and reduced morbidity and mortality for COVID-19: an epidemiological study

Stop playing with data: there is no sound evidence that Bacille Calmette-Guérin may avoid SARS-CoV-2 infection (for now)

Relation between BCG coverage rate and COVID-19 infection worldwide

Is BCG vaccination affecting the spread and severity of COVID-19

BCG vaccine protection from severe coronavirus disease 2019 (COVID-19)

Demystifying BCG Vaccine and COVID-19 Relationship

Regression Discontinuity Designs in Economics

Empirical Tests of the Validity of the Regression Discontinuity Design. Annales d'Économie et de

When does regression discontinuity design work? Evidence from random election outcomes

The impact of changing BCG coverage on tuberculosis incidence in Swedish-born children between 1969 and 1989

Regression discontinuity designs: A guide to practice

Robust Nonparametric Confidence Intervals for Regression-Discontinuity Designs

Robust Data-Driven Inference in the Regression-Discontinuity Design

What is meant by intention to treat analysis? Survey of published randomised controlled trials

Identification and Estimation of Local Average Treatment Effects

The BCG World Atlas: a database of global BCG vaccination policies and practices

Some evidence of the efficacy of mass BCG vaccination

Bacille Calmette-Guérin (BCG) vaccination and COVID-19

Global shortages of BCG vaccine and tuberculous meningitis in children