key: cord-0727114-ys6kfi15
authors: Szigeti, Reka; Kellermayer, Richard
title: Natural unblinding of BCG vaccination trials
date: 2021-03-18
journal: Vaccine
DOI: 10.1016/j.vaccine.2021.03.039
sha: 1cec12a881d4348af33c1b5c95c2ab2ce5106daf
doc_id: 727114
cord_uid: ys6kfi15

There is significant public and clinical interest in the potential for Bacillus Calmette-Guérin (BCG) vaccination to protect against type 2 Severe Acute Respiratory Syndrome Coronavirus (SARS-CoV-2) induced COVID-19. This question could be best answered by blinded and placebo controlled clinical trials. However, a skin reaction occurs within days at the site of BCG injection, making it rather challenging to blind this vaccination. Here, we examined registered clinical trials in ClinicalTrials.gov on BCG against COVID-19 by October 9th 2020, and found that 94.7% of such trials were listed as placebo controlled (all with normal saline as placebo), and single to quadruple blinded. The mode of overcoming the natural unblinding by the BCG induced skin reaction was not clarified on the website in either of the trials. We conclude that detailed description of the strategy towards overcoming the BCG vaccination induced skin reaction associated unblinding hurdle will be important for the interpretation of the theoretically blinded COVID-19 directed clinical trials.

A relatively large number of off-label Bacillus Calmette-Guerin (BCG) vaccination trials are registered, which aim to prevent type 2 Severe Acute Respiratory Syndrome Coronavirus (SARS-CoV-2) induced COVID-19. These trials recently received support by Evangelos J Giamarellos-Bourboulis et al. [1] , who performed an intriguing clinical study on BCG vaccination against off-target (i.e. nontuberculous) infections. The research group examined the effects of BCG (strain 1331; Intervax) vaccination compared to placebo (0.1 ml normal saline) on time interval to the first infection post hospital, and overall incidence of infections during a 12 month follow-up period in recently hospitalized elderly (>65y old) patients. Significant protection against respiratory tract infections of probable viral origin (hazard ratio 0.21, p: 0.013) was observed in the BCG group. The investigators connected these findings with plausible protection against SARS-CoV-2 by BCG vaccination, which exponentially heightens the impact of their results due to the ongoing COVID-19 pandemic. Such conclusion has been supported by Tomita, et al. [2] , who, through predictive modelling of human leukocyte antigen (HLA) class I-binding peptides, observed that cross-reactive T cells against SARS-CoV-2 could be generated by BCG vaccination.

Historical BCG vaccination and COVID-19 epidemiology derived conclusions about the long term, off target protection from this vaccination against SARS-CoV-2 are ambiguous [3] . Clinical observations and controlled trials, however, preceding the work of Giamarellos-Bourboulis, et al. indicated that acutely delivered BCG vaccination could potentially provide short term protection against off target bacterial and viral infections, especially in infants and children (reviewed in [4] ). However, there was only one placebo controlled trial published prior to 2020, which examined BCG from this off-target antiviral immunity perspective. Leentjens, et al. [5] , in a randomized double-blind placebo controlled trial, found that in 20 young adults, BCG vaccine significantly (p = 0.041) enhanced the immunogenicity of the 2009 pandemic influenza A (H1N1) vaccine, which administered 14 days after BCG vaccination, compared to 20 young adults who received normal saline as placebo. The group sample sizes, however, were low in this study allowing for false positive (type-1 error) and false negative (type-2 error) findings, which could have been corrected for with trial sequential analysis, for example.

Importantly, neither of the two publications on the placebo controlled trials listed above explained how double blinding (i.e. of patients and research personnel) could be achieved by using normal saline as placebo for BCG. Typically, a skin reaction occurs in 10-14 days at the site of the BCG injection and a permanent small scar develops in 3-6 months after the immunization in more than 85% of adults receiving the vaccine [6] . Therefore, it seems biologically not possible to blind BCG vaccination (especially with normal saline injection as placebo) in clinical investigations (see graphic abstract). Lack of appropriate blinding in randomized clinical trials may lead to significant bias [7] [8] [9] , although this dogma has been recently challenged [7, 10, 11] . Based on these considerations, we decided to investigate registered clinical trials in ClinicalTrials.gov on BCG vaccination against COVID-19.

We searched publically available data on trials registered in clinicaltrials.gov with words ''BCG" and ''COVID-19".

As of October 9, 2020, there were 19 registered clinical trials, which aimed to examine the acute protective effects of BCG vaccination against COVID-19 morbidity and/or mortality, over an observation period of 6 months following injection, most commonly. Eighteen (18/19; 94.7%) were placebo controlled (Table 1) , all with normal saline as placebo, and single to quadruple blinded per description. None of the trial narratives on the website clarified if and/or how exactly the researchers are planning to overcome the naturally occurring, dermal reaction induced biologic/physiologic unblinding of the subjects and/or research personnel.

We conclude that the large majority of registered clinical trials examining BCG's effect on COVID-19 are designed to be blinded, at least on the patient's side. However, due to a naturally occurring dermal reaction and scar, most subjects are bound to be unblinded within 2 weeks following injection (see graphic abstract).

The COVID-19 test can be frequently positive in asymptomatic individuals during outbreaks in vulnerable populations [12] . This indicates potentially high false positive rates of a single nasopharyngeal swab test for COVID-19. There is lack of a ''gold standard" test to which the results of the SARS-CoV-2 RT-PCR can be compared [13] . Consequently, it can be rather challenging to discern between truly uninfected states, asymptomatic pass-through (i.e. presence of passing through virus on mucosal surfaces without infection of host cells), asymptomatic infection/carriage, preclinical infection, or even respiratory symptoms arising from other infections in an individual with a SARS-CoV-2 positive PCR test. Under such circumstances, significant detection bias [14] may occur in a trial, if a naturally unblinded (as in case of BCG) subjects seek testing or are recommended to be tested (by the unblinded research personnel) differently than the control group. In case of BCG vaccination against COVID-19, the unblinded placebo group would likely have a higher rate of testing than the BCG group, which is ''intended" to be protected (intention bias and social desirability bias feeding into testing/detection bias). Additionally, the placebo group is likely to have a higher dropout rate from the study, leading to attrition bias [14] .Utilization of an ''ideal" nocebo (i.e., an immunologically inert substance that induces the Table 1 Registered clinical trials in clinical ClinicalTrials.gov as of October 9, 2020 identified with ''Covid19" and ''BCG" search words, which examine acutely delivered BCG vaccination against Covid-19 morbidity and mortality. Only one (italic) was not placebo controlled. All placebo-controlled trials designate normal saline as placebo, and are described as single to quadruple blinded. same dermal reaction as BCG vaccination, in this case) could provide the means to truly blind BCG vaccination trials. We could not find, however, any BCG trials by an internet search on January 3rd 2020 (with search terms on Google: BCG, vaccination, trial, nocebo) where such nocebo control would have been used.

If the natural unblinding of BCG vaccination is recognized, then prospectively set clinical criteria for testing and defining an a priori ''gold standard" set of tests (beyond a single nasopharyngeal PCR [as in [15] , for example]) for making the diagnosis of COVID-19 could counteract detection bias. Such measures could significantly improve the quality of an openly unblinded prospective clinical trial studying BCG against COVID-19. In the current status quo, however, detailed description of the strategy towards overcoming the BCG vaccination induced skin reaction associated unblinding, and the secondary biases, will be required to interpret the theoretically blinded COVID-19 directed clinical trials.

Discussion of biases holds true for the interpretation of retrospective observations on unblinded acute/booster BCG vaccination induced protection against COVID-19, as well. An example for such is the yet unpublished work of Amirlak, et al. [16] , which retrospectively found that 71 staff members of the Emirates International Hospital who received a BCG booster in an unblinded fashion were significantly (p = 0.004) protected against symptomatic COVID-19, compared to 209 staff members who did not receive the booster. It is unclear from the manuscript, however, how intention bias and social desirability bias in the booster group was controlled for (i.e., not seeking COVID-19 testing in case of mild symptomatology in the BCG booster group).

Activate: randomized clinical trial of BCG vaccination against Infection in the Elderly

BCG vaccine may generate cross-reactive T cells against SARS-CoV-2: In silico analyses and a hypothesis

BCG epidemiology supports its protection against COVID-19? A word of caution

The effect of Bacillus Calmette-Guerin (BCG) vaccination in preventing severe infectious respiratory diseases other than TB: implications for the COVID-19 pandemic

BCG vaccination enhances the immunogenicity of subsequent influenza vaccination in healthy volunteers: A randomized. Placebo-Controlled Pilot Study

BCG scar and tuberculin reactivity in children and adults. Scand

Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies

Observer bias in randomized clinical trials with time-to-event outcomes: systematic review of trials with both blinded and non-blinded outcome assessors

Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors

Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study

Blindsided: challenging the dogma of masking in clinical trials

Lessons from mass-testing for COVID-19 in long term care facilities for the elderly in San Francisco

Falsepositive reverse transcriptase polymerase chain reaction screening for SARS-CoV-2 in the setting of urgent head and neck surgery and otolaryngologic emergencies during the pandemic: clinical implications

Biases in randomized trials: a conversation between trialists and epidemiologists

Virological assessment of hospitalized patients with COVID-2019

Effectiveness of booster BCG vaccination in preventing Covid-19 infection

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.