key: cord-314121-ez62fx9p authors: de Chaisemartin, C.; de Chaisemartin, L. title: BCG vaccination in infancy does not protect against COVID-19. Evidence from a natural experiment in Sweden. date: 2020-06-23 journal: nan DOI: 10.1101/2020.06.22.20137802 sha: doc_id: 314121 cord_uid: ez62fx9p Background The Bacille Calmette-Guerin (BCG) tuberculosis vaccine has immunity benefits against respiratory infections. Accordingly, it has been hypothesized that it may have a protective effect against COVID-19. Recent research found that countries with universal BCG childhood vaccination policies tend to be less affected by the COVID-19 pandemic. However, such ecological studies are biased by numerous confounders. Instead, this paper takes advantage of a rare nationwide natural experiment that took place in Sweden in 1975, where discontinuation of newborns BCG vaccination led to a dramatic fall of the BCG coverage rate from 92% to 2% , thus allowing us to estimate the BCG's effect without all the biases associated with cross-country comparisons. Methods Numbers of COVID-19 cases and hospitalizations were recorded for birth cohorts born just before and just after that change, representing 1,026,304 and 1,018,544 individuals, respectively. We used regression discontinuity to assess the effect of BCG vaccination on Covid-19 related outcomes. This method used on such a large population allows for a high precision that would be hard to achieve using a classical randomized controlled trial. Results The odds ratio for Covid-19 cases and Covid-19 related hospitalizations were 0.9997 (CI95: [0.8002-1.1992]) and 1.1931 (CI95: [0.7558-1.6304]), respectively. We can thus reject with 95% confidence that universal BCG vaccination reduces the number of cases by more than 20% and the number of hospitalizations by more than 24% Conclusions While the effect of a recent vaccination must be evaluated, we provide strong evidence that receiving the BCG vaccine at birth does not have a protective effect against COVID-19. The Bacille Calmette-Guérin (BCG) tuberculosis vaccine has immunity benefits against non-targeted pathogens 1 , and in particular against respiratory infections caused by RNA viruses like influenza 2 . Since SARS-Cov-2 is also a single-stranded RNA virus, it has been hypothesized that differences in BCG vaccination coverage could explain the wide differences in disease burden observed between countries. A pioneering preprint paper by Miller et al. found that countries with universal Bacillus Calmette-Guérin (BCG) childhood vaccination policies tend to be less affected by the COVID-19 pandemic, in terms of their number of cases and deaths 3 . While unpublished, this study had a great impact and gave rise to many comments and follow-up studies (reviewed in 4 ). Some published studies were able to replicate this result 5, 6 , but several authors underlined the important statistical flaws inherent to such ecological studies and concluded that randomized controlled trials (RCT) were necessary to address the question 4,7 . As of June 5th 2020, no less than 12 randomized controlled trials (RCTs) studying the protective effect of the BCG against COVID-19 are already registered on https://clinicaltrials.gov/. However, none has a primary completion date earlier than October 1 st 2020, so these RCTs' first results will not be known until at least five or six months. With the epidemic still on the rise worldwide, and in the absence of a SARS-Cov-2 vaccine, there is an urgent need to know if BCG non-specific effects could be harnessed as a substitute prophylactic treatment. Regression discontinuity (RD) is a method designed by social scientists to assess the effect of an exposure on an outcome. It is deemed as reliable as RCTs to tease out causality from correlation 8 , and typically yields results similar to those obtained in RCTs 9, 10 . In this paper, we applied this method to a rare natural experiment that took place in Sweden. Sweden currently has the 5th highest ratio of COVID-19 deaths per capita in the world. In April 1975, it stopped its newborns BCG vaccination program, leading to a dramatic drop of the BCG vaccination rate from 92% to 2% for cohorts born just before and just after the All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 23, 2020. . https://doi.org/10.1101/2020.06.22.20137802 doi: medRxiv preprint change 11 . We compared the number of COVID-19 cases, hospitalizations, and deaths per capita, for cohorts born just before and just after April 1975, representing 1,026,304 and 1,018,544 individuals, respectively. Using RD, we were able to show that those cohorts do not have different numbers of COVID-19 cases, hospitalizations or deaths per capita, with a high precision that would hardly be possible to reach with a RCT design. Regression discontinuity (RD) is a commonly-used method to measure the effect of a treatment on an outcome 13 . It is applicable when only individuals that satisfy a strict criterion are eligible for a policy. Then, RD amounts to comparing the outcome of interest among individuals just above and just below the eligibility threshold. In this study, RD will amount to comparing the number of COVID-19 cases, hospitalizations, and deaths among individuals born just before and just after April 1st 1975. The effect All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 23, 2020. . https://doi.org/10.1101/2020.06.22.20137802 doi: medRxiv preprint of universal BCG vaccination for individuals born around April 1st, 1975 was estimated using the stateof-the-art estimator for RD 14 . The estimator amounts to comparing treated and control units, in a narrow window around April 1st 1975. It uses linear regressions of the outcome on birth cohort to the left and to the right of the threshold, to predict the outcome of treated and untreated units at the threshold. Then, the estimator is the difference between these two predicted values. The estimator and 95% confidence interval were computed using the rdrobust Stata command, see 15 . This study uses the number of COVID-19 cases per 1000 inhabitants for quarterly birth cohorts born between Q1-1930 and Q4-2001, the number of COVID-19 hospitalizations per 1000 inhabitants for cohorts born between Q1-1930 and Q4-1991, and the number of COVID-19 deaths per 1000 inhabitants for groups of three yearly birth cohorts, from 1930-1931-1932 to 1978-1979-1980 . These variables were constructed using data compiled by the Public Health Agency of Sweden; see the supplementary Table 1 for details. In an RD design, the presence or absence of a treatment effect can be assessed visually, by drawing a scatter-plot with the variable determining eligibility on the x-axis, and the outcome variable on the y-axis. If one observes that the relationship between these two variables jumps discontinuously at the eligibility threshold, this is indicative of a treatment effect. Accordingly, Figure 1 shows no discontinuity in the numbers of COVID-19 cases per 1000 inhabitants for cohorts born just before and just after April 1975. This suggests that universal BCG vaccination has no effect on the number of COVID-19 cases per 1000 inhabitants for individuals born in 1975. Figures 2 and 3 show that similar conclusions apply when one looks at the number of COVID-19 hospitalizations per 1000 inhabitants and at the number of COVID-19 deaths per 1000 inhabitants. The number of deaths per 1000 inhabitants is several orders of magnitudes All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 23, 2020. . https://doi.org/10.1101/2020.06.22.20137802 doi: medRxiv preprint higher for older than for younger cohorts, so Figure 3 only presents those numbers for cohorts born after 1960 to keep the graph legible. This visual analysis is confirmed by the statistical calculations. Table 1 inhabitants, there is only two data points to the right of the Q2-1975 threshold. Therefore, the RD estimator cannot be computed for that outcome. Instead, we just compared the number of COVID-19 deaths per 1000 inhabitants in the 1972-1973-1974 and 1975-1976-1977 YBCs using a standard t-test, even though this method does not account for the fact those two groups differ in age, contrary to the RD method. Doing so, we find that the difference between the deaths per 1000 inhabitants of the two groups is not different from 0. Table 1 are intention-to-treat effects 16 : not all Swedish residents born just before April 1975 received the BCG vaccine at birth, and some of those born just after April 1975 received it. In particular, foreign-born residents account for 27•2% of the Swedish population born in 1975 as per Statistics Sweden's data, and they were not affected by the 1975 policy. Among natives, the policy led to a drop of the vaccination rate from 92 to 2% 11 . Assuming that the BCG vaccination rate of foreign-born residents is the same just before and just after April 1975, a reasonable assumption as no other European country changed its BCG vaccination policy in 1975, the policy led to a drop in the BCG vaccination rate All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 23, 2020. Table 1 into the effect of being vaccinated at birth, one needs to divide the intention-to-treat effects and their confidence intervals by 0•655, the change in the BCG vaccination rate at birth induced by the reform 17 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 23, 2020. In this study, we took advantage of a change in vaccination policy in Sweden to investigate the link between BCG vaccination in infancy and Covid-19 cases, hospitalizations and deaths, using a regression discontinuity approach. Contrarily to most studies on the question, we compared Covid-19 cases between two very similar groups of people from the same country. This allows us to get rid of all confounders linked to cross-countries comparisons, and of confounders like sex or socio-economics status that are often present in observational studies that do not rely on a quasi-experimental design, unlike ours. Another strength of this study is its statistical precision. Since we could gather nationwide data in a country where Covid-19 rates are high, we are able to reject fairly small effects of the BCG vaccine. Achieving a comparable statistical precision in an RCT would require an unrealistically large sample. Even with a COVID-19 hospitalization rate of 0•5%, as among the elderly Swedish population, a randomized controlled trial that could reject BCG effects larger than 24% of the baseline hospitalization rate, as in our study, would require including around 15,000 participants. While previous studies mostly addressed differences in BCG vaccination policies but did not account for differences in actual BCG coverage, we work with two populations with well documented and very 11 . Prior to that, Sweden had already eliminated its revaccination program at 7 years of age in 1965. Sweden also stopped its revaccination All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 23, 2020. Moreover, this study does not measure the COVID-19 immunity benefit conferred by a recent BCG vaccination, as individuals born just before Q2-1975 were vaccinated 45 years ago. The RCTs currently All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 23, 2020. . https://doi.org/10.1101/2020.06.22.20137802 doi: medRxiv preprint underway will tell if the protective effect of a recent BCG vaccination differs from the effect measured in this study. Overall, this study shows BCG vaccination at birth does not have a strong protective effect against COVID-19. Thus, it seems that BCG childhood vaccination policies cannot account for the differences in the severity of the pandemic across countries, as had been hypothesized by prior studies 3,5,6 . This advocates for a strict adherence to WHO's recommendation of the vaccine to infants outside of clinical trials 21 , and for restraint from starting new clinical trials on this question. The question is of particular importance for a vaccine whose lengthy production process regularly leads to worldwide shortages with dire consequences on children from country with high prevalence of tuberculosis 22 . While RCTs will complement this study by measuring the effect of a recent vaccination, this study comes much before results of the RCTs will be made available, and with a greater precision. Finally, it exemplifies the potential of leveraging past medical policies and statistical techniques designed in the social sciences to answer current medical questions. (2), and the 95% confidence interval of this effect is shown in Column (3) All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 23, 2020. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 23, 2020. 1960-1961-1962 to birth cohorts 1978-1979-1980. 1975 , when vaccination at birth was discontinued, is represented by the red vertical line. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 23, 2020. . https://doi.org/10.1101/2020.06.22.20137802 doi: medRxiv preprint Notes: Some QBCs after Q4-1991 had less than 5 hospitalizations, so Folkhälsomyndigheten could not provide their number of hospitalizations due to confidentiality issues. The Swedish population per QBC is not publicly available, while the population per YBC is. Rather than just dividing the YBC's population by four to recover each QBC's population, we account for the fact that Sweden exhibits a little bit of quarterly birth seasonality. From 1930 to 2001, 25•6%, 27•0%, 24•9%, and 22•5% of births respectively happened during Q1, Q2, Q3, and Q4. To infer a QBC's population, we multiply the population of the corresponding YBC by the proportion this quarter accounts for in the births that took place that year in Sweden. Not all Swedish inhabitants were born in Sweden, so this computation implicitly assumes that quarterly birth seasonality is the same in Sweden as in the countries foreign-born-Sweden-residents immigrated from. Foreign-born individuals only account for 22% of the 1930 to 2001 YBC Swedish population, and quarterly birth seasonality is weak in most countries, so this assumption should not strongly affect the results. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 23, 2020. . https://doi.org/10.1101/2020.06.22.20137802 doi: medRxiv preprint The non-specific and sex-differential effects of vaccines Non-specific effects of BCG vaccine on viral infections Relation between BCG coverage rate and COVID-19 infection worldwide Is BCG vaccination affecting the spread and severity of COVID-19? Demystifying BCG Vaccine and COVID-19 Relationship Regression Discontinuity Designs in Economics Empirical Tests of the Validity of the Regression Discontinuity Design When does regression discontinuity design work? Evidence from random election outcomes The impact of changing BCG coverage on tuberculosis incidence in Swedish-born children between 1969 and 1989 Regression discontinuity designs: A guide to practice Robust Nonparametric Confidence Intervals for Regression-Discontinuity Designs Robust Data-Driven Inference in the Regression-Discontinuity Design What is meant by intention to treat analysis? Survey of published randomised controlled trials Identification and Estimation of Local Average Treatment Effects The BCG World Atlas: a database of global BCG vaccination policies and practices Some evidence of the efficacy of mass BCG vaccination Bacille Calmette-Guérin (BCG) vaccination and COVID-19 Global shortages of BCG vaccine and tuberculous meningitis in children We are grateful to Folkhälsomyndigheten, the Public Health Agency of Sweden, for providing the data used in this study and answering all our questions. We are also grateful to Martin Berlin, JohannesHaushofer, Jerker Jonsson, Ellen Lundqvist, Kyle Meng, Robert Östling, Andrew Oswald, Moa Rehn, and Gonzalo Vazquez-Bare for their help.All rights reserved. No reuse allowed without permission.(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.The copyright holder for this preprint this version posted June 23, 2020. .