key: cord-208177-m8q6dyfx
authors: Chaisemartin, Cl'ement de; Chaisemartin, Luc de
title: BCG vaccination in infancy does not protect against COVID-19. Evidence from a natural experiment in Sweden
date: 2020-06-08
journal: nan
DOI: nan
sha: 
doc_id: 208177
cord_uid: m8q6dyfx

The Bacille Calmette-Gu'erin (BCG) tuberculosis vaccine has immunity benefits against respiratory infections. Accordingly, it has been hypothesized that it may have a protective effect against COVID-19. Recent research found that countries with universal Bacillus Calmette-Gu'erin (BCG) childhood vaccination policies tend to be less affected by the COVID-19 pandemic. However, such ecological studies are biased by numerous confounders. Instead, this paper takes advantage of a rare nationwide natural experiment that took place in Sweden in 1975, where discontinuation of newborns BCG vaccination led to a dramatic fall of the BCG coverage rate from 92% to 2% , thus allowing us to estimate the BCG's effect without all the biases associated with cross-country comparisons. Numbers of COVID-19 cases and hospitalizations were recorded for birth cohorts born just before and just after that change, representing 1,026,304 and 1,018,544 individuals, respectively. We used regression discontinuity to assess the effect of BCG vaccination on Covid-19 related outcomes. This method used on such a large population allows for a high precision that would be hard to achieve using a classical randomized controlled trial. The odds ratio for Covid-19 cases and Covid-19 related hospitalizations were 0.9997 (CI95: [0.8002-1.1992]) and 1.1931 (CI95: [0.7558-1.6304]), respectively. We can thus reject with 95% confidence that universal BCG vaccination reduces the number of cases by more than 20% and the number of hospitalizations by more than 24%. While the effect of a recent vaccination must be evaluated, we provide strong evidence that receiving the BCG vaccine at birth does not have a protective effect against COVID-19.

The Bacille Calmette-Guérin (BCG) tuberculosis vaccine has immunity benefits against non-targeted pathogens 1 , and in particular against respiratory infections caused by RNA viruses like influenza 2 . Since SARS-Cov-2 is also a single-stranded RNA virus, it has been hypothesized that differences in BCG vaccination coverage could explain the wide differences in disease burden observed between countries.

A pioneering preprint paper by Miller et al. found that countries with universal Bacillus Calmette-Guérin (BCG) childhood vaccination policies tend to be less affected by the COVID-19 pandemic, in terms of their number of cases and deaths 3 . While unpublished, this study had a great impact and gave rise to many comments and follow-up studies (reviewed in 4 ). Some published studies were able to replicate this result 5, 6 , but several authors underlined the important statistical flaws inherent to such ecological studies and concluded that randomized controlled trials (RCT) were necessary to address the question 4,7 . As of June 5th 2020, no less than 12 randomized controlled trials (RCTs) studying the protective effect of the BCG against COVID-19 are already registered on https://clinicaltrials.gov/. However, none has a primary completion date earlier than October 1 st 2020, so these RCTs' first results will not be known until at least five or six months. With the epidemic still on the rise worldwide, and in the absence of a SARS-Cov-2 vaccine, there is an urgent need to know if BCG non-specific effects could be harnessed as a substitute prophylactic treatment.

Regression discontinuity (RD) is a method designed by social scientists to assess the effect of an exposure on an outcome. It is deemed as reliable as RCTs to tease out causality from correlation 8 , and typically yields results similar to those obtained in RCTs 9,10 . In this paper, we applied this method to a rare natural experiment that took place in Sweden. Sweden currently has the 5th highest ratio of COVID-19 deaths per capita in the world. In April 1975, it stopped its newborns BCG vaccination program, leading to a dramatic drop of the BCG vaccination rate from 92% to 2% for cohorts born just before and just after the change 11 . We compared the number of COVID-19 cases, hospitalizations, and deaths per capita, for cohorts born just before and just after April 1975, representing 1,026,304 and 1,018,544 individuals, respectively. Using RD, we were able to show that those cohorts do not have different numbers of COVID-19 cases, hospitalizations or deaths per capita, with a high precision that would hardly be possible to reach with a RCT design. 

Regression discontinuity (RD) is a commonly-used method to measure the effect of a treatment on an outcome 13 . It is applicable when only individuals that satisfy a strict criterion are eligible for a policy.

Then, RD amounts to comparing the outcome of interest among individuals just above and just below the eligibility threshold. In this study, RD will amount to comparing the number of COVID-19 cases, hospitalizations, and deaths among individuals born just before and just after April 1st 1975. The effect of universal BCG vaccination for individuals born around April 1st, 1975 was estimated using the stateof-the-art estimator for RD 14 . The estimator amounts to comparing treated and control units, in a narrow window around April 1st 1975. It uses linear regressions of the outcome on birth cohort to the left and to the right of the threshold, to predict the outcome of treated and untreated units at the threshold.

Then, the estimator is the difference between these two predicted values. The estimator and 95% confidence interval were computed using the rdrobust Stata command, see 15 .

This study uses the number of COVID-19 cases per 1000 inhabitants for quarterly birth cohorts born between Q1-1930 and Q4-2001, the number of COVID-19 hospitalizations per 1000 inhabitants for cohorts born between Q1-1930 and Q4-1991, and the number of COVID-19 deaths per 1000 inhabitants for groups of three yearly birth cohorts, from 1930-1931-1932 to 1978-1979-1980 . These variables were constructed using data compiled by the Public Health Agency of Sweden; see the supplementary Table 1 for details.

In an RD design, the presence or absence of a treatment effect can be assessed visually, by drawing a scatter-plot with the variable determining eligibility on the x-axis, and the outcome variable on the y-axis.

If one observes that the relationship between these two variables jumps discontinuously at the eligibility threshold, this is indicative of a treatment effect. Accordingly, Figure 1 shows no discontinuity in the numbers of COVID-19 cases per 1000 inhabitants for cohorts born just before and just after April 1975.

This suggests that universal BCG vaccination has no effect on the number of COVID-19 cases per 1000 This visual analysis is confirmed by the statistical calculations. Table 1 inhabitants, there is only two data points to the right of the Q2-1975 threshold. Therefore, the RD estimator cannot be computed for that outcome. Instead, we just compared the number of COVID-19 deaths per 1000 inhabitants in the 1972-1973-1974 and 1975-1976-1977 YBCs using a standard t-test, even though this method does not account for the fact those two groups differ in age, contrary to the RD method. Doing so, we find that the difference between the deaths per 1000 inhabitants of the two groups is not different from 0. Table 1 are intention-to-treat effects 16 : not all Swedish residents born just before April 1975 received the BCG vaccine at birth, and some of those born just after April 1975 received it. In particular, foreign-born residents account for 27•2% of the Swedish population born in 1975 as per Statistics Sweden's data, and they were not affected by the 1975 policy. Among natives, the policy led to a drop of the vaccination rate from 92 to 2% 11 . Assuming that the BCG vaccination rate of foreign-born residents is the same just before and just after April 1975, a reasonable assumption as no other European country changed its BCG vaccination policy in 1975, the policy led to a drop in the BCG vaccination rate of 0•655 ((1-0•272)×(0•92-0•02)=0•655). Then, to convert the intention-to-treat effects in Table 1 into the effect of being vaccinated at birth, one needs to divide the intention-to-treat effects and their confidence intervals by 0•655, the change in the BCG vaccination rate at birth induced by the reform 17 

In this study, we took advantage of a change in vaccination policy in Sweden to investigate the link between BCG vaccination in infancy and Covid-19 cases, hospitalizations and deaths, using a regression discontinuity approach.

Contrarily to most studies on the question, we compared Covid-19 cases between two very similar groups of people from the same country. This allows us to get rid of all confounders linked to cross-countries comparisons, and of confounders like sex or socio-economics status that are often present in observational studies that do not rely on a quasi-experimental design, unlike ours. Another strength of this study is its statistical precision. Since we could gather nationwide data in a country where Covid-19

rates are high, we are able to reject fairly small effects of the BCG vaccine. Achieving a comparable statistical precision in an RCT would require an unrealistically large sample. Even with a COVID-19 hospitalization rate of 0•5%, as among the elderly Swedish population, a randomized controlled trial that could reject BCG effects larger than 24% of the baseline hospitalization rate, as in our study, would require including around 15,000 participants.

While previous studies mostly addressed differences in BCG vaccination policies but did not account for differences in actual BCG coverage, we work with two populations with well documented and very Moreover, this study does not measure the COVID-19 immunity benefit conferred by a recent BCG vaccination, as individuals born just before Q2-1975 were vaccinated 45 years ago. The RCTs currently underway will tell if the protective effect of a recent BCG vaccination differs from the effect measured in this study.

Overall, this study shows BCG vaccination at birth does not have a strong protective effect against COVID-19. Thus, it seems that BCG childhood vaccination policies cannot account for the differences in the severity of the pandemic across countries, as had been hypothesized by prior studies 3,5,6 . This advocates for a strict adherence to WHO's recommendation of the vaccine to infants outside of clinical trials 21 , and for restraint from starting new clinical trials on this question. The question is of particular importance for a vaccine whose lengthy production process regularly leads to worldwide shortages with dire consequences on children from country with high prevalence of tuberculosis 22 .

While RCTs will complement this study by measuring the effect of a recent vaccination, this study comes much before results of the RCTs will be made available, and with a greater precision. Finally, it exemplifies the potential of leveraging past medical policies and statistical techniques designed in the social sciences to answer current medical questions. 1978-1979-1980. 1975 , when vaccination at birth was discontinued, is represented by the red vertical line. 

The non-specific and sex-differential effects of vaccines

Relation between BCG coverage rate and COVID-19 infection worldwide

Is BCG vaccination affecting the spread and severity of COVID-19?

Demystifying BCG Vaccine and COVID-19 Relationship

Regression Discontinuity Designs in Economics

Empirical Tests of the Validity of the Regression Discontinuity Design

When does regression discontinuity design work? Evidence from random election outcomes

The impact of changing BCG coverage on tuberculosis incidence in Swedish-born children between 1969 and 1989

Regression discontinuity designs: A guide to practice

Robust Nonparametric Confidence Intervals for Regression-Discontinuity Designs

Robust Data-Driven Inference in the Regression-Discontinuity Design

What is meant by intention to treat analysis? Survey of published randomised controlled trials

Identification and Estimation of Local Average Treatment Effects

The BCG World Atlas: a database of global BCG vaccination policies and practices

Duration of BCG protection against tuberculosis and change in effectiveness with time since vaccination in Norway: a retrospective population-based cohort study

Some evidence of the efficacy of mass BCG vaccination

Bacille Calmette-Guérin (BCG) vaccination and COVID-19

Global shortages of BCG vaccine and tuberculous meningitis in children

We are grateful to Folkhälsomyndigheten, the Public Health Agency of Sweden, for providing the data used in this study and answering all our questions. We are also grateful to Martin Berlin, JohannesHaushofer, Jerker Jonsson, Ellen Lundqvist, Kyle Meng, Robert Östling, Andrew Oswald, Moa Rehn, and Gonzalo Vazquez-Bare for their help.