key: cord-0923925-6hebzuww authors: Hemkens, Lars G.; Goodman, Steven N. title: Randomized COVID-19 vaccination rollout can offer direct real-world evidence: Randomized COVID-19 vaccination rollout date: 2021-05-25 journal: J Clin Epidemiol DOI: 10.1016/j.jclinepi.2021.05.014 sha: 59c0aa23a95bbf1630269b446f071400e87cdeb4 doc_id: 923925 cord_uid: 6hebzuww Vaccines are vital to control the Coronavirus disease 2019 (COVID-19) pandemic, but the pressure to quickly move from research to implementation in the context of a pandemic crisis raises concerns about benefits and harms, vaccine acceptance and fair access. Here we present a strategy for the COVID-19 vaccination rollout which can be rapidly embedded and would offer direct real-world evidence of vaccines on a large scale to generate otherwise unobtainable knowledge on the safety and perhaps efficacy of COVID-19 vaccines. Such strategic rollouts leveraging randomization can provide important evidence, for COVID-19 and in future occasions, for vaccines and beyond.  Vaccines are vital to control the Coronavirus disease 2019 (COVID-19) pandemic, but the pressure to quickly move from research to implementation in the context of a pandemic crisis raises concerns about benefits and harms, vaccine acceptance and fair access.  A randomized strategy for the COVID-19 vaccination rollout can be rapidly embedded in the planning and regulatory organization of rollout procedures and would offer direct realworld evidence on a large scale to generate otherwise unobtainable knowledge on the safety and perhaps efficacy.  Strategic rollouts leveraging randomization provide important evidence for vaccines and for other large-scale health interventions in the future. Vaccines are vital to control the Coronavirus disease 2019 (COVID-19) pandemic, but the pressure to quickly move from research to implementation in the context of a pandemic crisis raises concerns about benefits and harms, vaccine acceptance and fair access. The randomized clinical trials (RCTs) supporting approval of the first two vaccines include roughly 30,000-40,000 participants, provide safety data gathered over a period of at least 8 weeks, but have too few clinical events to assess outcomes like mortality, hospitalizations, and serious or rare adverse events 1,2 . Only 6 serious COVID-19 cases were reported for the Pfizer-BioNTech and Moderna trials combined 1,2 . Subgroup effects or drug-interactions are unlikely to be detectable. However, after approval, trials with no-treatment or placebo controls and representative samples are unlikely to be feasible 3, 4 , and in both current trials, the length of further observation of the placebo controls is being curtailed 5 . But even when placebo-controlled trials can no longer be performed nearly equivalent information can be generated from strategic vaccine rollout. In situations when all priority factors are equal and no relevant differences exist among potential recipients of the vaccination, a classic way to ensure fair access would be a random procedure. In 2008, Medicaid expansion in Oregon was implemented by using random drawing from a waiting list to provide uninsured low-income adults the chance to apply for Medicaid. Designed as experiment, this process not only ensured fairness, it also provided a unique opportunity for generating highquality evidence: studies showed, for example, that Medicaid coverage did not significantly improve physical health in the short-term but increased healthcare utilization, changed condition-specific treatments and reduced depression incidence 6 . An equivalent strategy could still be used for COVID-19 vaccination, taking advantage of the prolonged rollout in some countries to create the equivalent of otherwise infeasible placebocontrolled trials. Individual position on prioritization lists could be randomly determined, e.g., the number of days to wait before vaccination is offered. Alternatively, if the position on such lists were effectively random, analyses could utilize the quasi-random nature of immunization timing. The incidence of outcomes can be compared between those who have been vaccinated earlier and those who are still waiting. Places on waiting list are already provided in random order (within priority groups) in some areas. Minnesota, for example, allocates some appointments based on randomization 7 , as does the German federal state Saarland with its population of 1 million people 8 . Information on severe outcomes is collected routinely for healthcare or COVID-related public surveillance. Administrative claims and pharmacy data capture severe events requiring hospitalizations or costly procedures. While specific risk groups are often excluded from most approval studies (such as immunocompromised or pregnant patients), they may be included in community vaccination plans, pregnancy care programs, or even disease registries, which routinely collect data of special importance for these groups. Routinely collected data are widely used for outcome collection in randomized clinical trials 9 . Data quality, privacy issues, and misclassification biases need to be considered very carefully 10 , but this is true regardless of whether data are used for surveillance or studies with or without randomization. However, randomization would ensure that misclassification biases are towards the null (as long as data quality is stable during follow-up); poor data quality would introduce noise equally in both comparison groups and could dilute potential differences so that safety signals are missed, but it would not trigger false alarms. Even if such randomized rollout was limited to only a single small jurisdiction or integrated healthcare organization allowing to collect the outcomes 11 , the data on vaccines would likely be many-fold larger than all clinical evidence available at approval. The population would have maximal representativeness and be large enough to detect or rule out rare serious side effects occurring early, with randomized, or quasi-randomized control groups that observational pharmacosurveillance studies often struggle to construct. In a country or state there could easily be 3 million individuals with equal priority factors and no relevant differences related to COVID-19 (e.g. in the same age group without specific risk factors). If logistics and vaccine availability permitted vaccination of such a group within 150 days, vaccination of 20,000 persons per day would be needed. Comparing event rates between those invited for vaccination on (for example) the first five days vs those invited for the last five days would be the equivalent to a conventional RCT with 200,000 participants and median follow-up of about 21 weeks (150 days). The comparison would be between the two groups in the same first calendar window (e.g. by providing risk ratios with confidence intervals), on both efficacy and safety outcomes derived from routinely collected data, the same currently being used for population-based pharmacovigilance studies. The waiting time between the two vaccination periods would constitute the effective duration of this randomized experiment. In clinical medicine, waiting list randomization is common when a group without treatment would not be acceptable. WHO experts recently proposed this design for COVID-19 12 . Stepped wedge trials, a special form of cluster randomization, employ dynamic processes with successive introduction and implementation of system innovations on a group level. In one stepped-wedge trial in Serrana (Brazil), 27,619 adults were vaccinated against COVID-19 within four weeks in February/March 2021 13,14 . The town Serrana was divided in four clusters, which were randomly allocated to one of four subsequent weeks. In each week, all enrolled adults in the cluster were offered the first shot of the vaccine. In the next week, the next cluster was vaccinated. After four weeks, all enrolled adults received the first vaccination and comparison of observed events across clusters is possible. An individually randomized rollout model described here would resemble an individual stepped wedge design where not entire clusters but individual patients are randomized. It has been described as "dynamic waiting list randomization" 15 for evaluation of public health interventions, with well-developed statistical frameworks 16 . This idea could be implemented using the lists of all persons eligible for vaccination that are generated by authorities, health insurers, or employers (e.g. hospitals, schools, nursing homes). A random priority sequence would be generated, and each listed person is allocated to one time-point and receives a unique person identifier. Second, information about vaccinations received would be collected centrally in a vaccination registry, only recording for each person the date of vaccination. Third, information on events of interest would be required. COVID-19 diagnoses are routinely collected in most countries. Data on hospitalizations may be available routinely for all or most patients in some jurisdictions. If not, participation in a rollout program could be routinely recorded for all patients hospitalized during the rollout period. Digital apps may offer further options to collect outcomes when combined with the rollout process. Health care utilization and economic outcomes could be assessed as well. Finally, rollout list entries are linked via the patient identifier with the vaccine registry and the outcome data sources. Since most or all data collection infrastructures already exist, no extra time or resources are needed before the rollout; data linkage, processing, cleaning can be done afterwards. There are several challenges. First, if immunization of equivalent cohorts is not separated sufficiently in time, then there is not enough untreated follow-up time to compare. Second, as this is unblinded, all the problems of selective reporting of symptoms or signs or differential risk behavior arise. Third, a randomized invitation requires some process modifications but this unlikely slows down the vaccination process overall. The feasibility is underlined by the fact that this procedure has already been used, but not for the purpose of science 7, 8 . Fourth, issues of consent need to be clarified 10 , but participants are not exposed to additional health risks because the approach would not affect the average waiting time, and most data are routinely collected. Fifth, not all persons will get vaccinated according to the prioritization list. The very large samples may then provide some compensation by increased precision and carefully planned causal modelling approaches may offer additional help. Finally, if vaccination and outcome status cannot be linked via a patient identifier, then this cannot be implemented. In the US, this means that it may be most feasible within closed health systems (e.g. Kaiser, Geisinger, Intermountain Health) or in established structures (e.g. VSD). Despite all these limitations, randomized or quasi-randomized vaccinated versus non-vaccinated contrasts would still facilitate much better control of biases than currently proposed designs such as uncontrolled pharmacovigilance studies, case-control or modelling approaches, and would not require collecting all the data needed for confounder adjustment. The current uncertainty about potential adverse events related to the AstraZeneca vaccine 17 illustrates the need for solid evidence to assess safety signals. A randomized rollout strategy would inform such discussions by helping to clarify if such observations are causally related to the vaccine. This approach would be useful for COVID-19 as long as there is a rollout of vaccines and members of the target populations are waiting to receive them. Strategic rollouts leveraging randomization can provide important evidence for COVID-19 vaccines and for other large-scale health interventions in the future. Overall, a randomized or quasi-randomized rollout process would offer direct real-world evidence of vaccines and can be rapidly embedded in the planning and regulatory organization of rollout procedures, on a large scale to generate otherwise unobtainable knowledge on the safety and perhaps efficacy of COVID-19 vaccines. All authors attest they meet the ICMJE criteria for authorship and that no others meeting the criteria have been omitted. Disclosure of potential conflicts of interest: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf. There are no relationships or activities that could appear to have influenced the submitted work. FDA Briefing Document. Moderna COVID-19 Vaccine. Vaccines and Related Biological Products Advisory Committee Meeting FDA Briefing Document. Pfizer-BioNTech COVID-19 Vaccine. Vaccines and Related Biological Products Advisory Committee Meeting COVID-19 vaccine trial ethics once we have efficacious vaccines New placebo-controlled Covid-19 vaccine trials are ethically questionable; it's now about comparative effectiveness and availability of registered vaccines Makers of successful COVID-19 vaccines wrestle with options for placebo recipients The Oregon experiment--effects of Medicaid on clinical outcomes Minnesota COVID-19 Response. COVID-19 Community Vaccination Program Treatment effects in randomised trials using routinely collected data for outcome assessment versus traditional trials: meta-research study Routinely collected data for randomized trials: promises, barriers, and implications The Vaccine Safety Datalink: successes and challenges monitoring vaccine safety NCT04747821: An Effectiveness Study of the Sinovac's Adsorbed COVID-19 (Inactivated) Vaccine (Projeto S) Dynamic wait-listed designs for randomized trials: new designs for prevention of youth suicide Adaptive Designs for Randomized Trials in Public Health EMA's safety committee continues investigation of COVID-19