key: cord-1012470-sazjjvsz authors: Jiang, Zhiwei; Wang, Xuanyi; Xia, Jielai title: Considerations on the clinical development of COVID-19 vaccine from trial design perspectives date: 2020-09-29 journal: Human vaccines & immunotherapeutics DOI: 10.1080/21645515.2020.1815489 sha: 41484956bafe260446a252e7ed3018b507832bdb doc_id: 1012470 cord_uid: sazjjvsz COVID-19 has become a global pandemic, and an effective vaccine is needed. During the outbreak, the urgency for developing candidate vaccines has brought distinct challenges to clinical development. An efficacy trial, which measures whether the vaccine reduces the incidence of disease, is ordinarily required to fully evaluate vaccine efficacy. However, emergency use may be possible if promising immunogenicity results are observed. A ring vaccination trial, which recruits subjects connected to a known case either socially or geographically, is a solution to evaluate vaccine efficacy and control the spread of the disease simultaneously although its conduct is challenging. Nevertheless, when COVID-19 becomes a recurrent epidemic, an ‘individual-level’ efficacy trial is preferred. Innovative statistical designs, including seamless design, platform trial, master protocol design, are helpful to accelerate clinical development. A seamless Phase I/II design has been applied in multiple COVID-19 vaccine studies to date. However, Phase II/III design should be done very carefully. The control of type I error, maintaining trial blinding and statistical methods leading to unbiased estimates should be pre-specified in the clinical protocol. A Data Safety Monitoring Board is especially important, given the need to assure an adequate level of safety when society want a safe and effective vaccine. Coronavirus disease (COVID- 19) is an emerging infectious disease that was first reported at the end of 2019. The disease has evolved into a pandemic that is well known for being highly contagious and lethal. According to statistics from the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University, 1 COVID-19 infections have occurred in almost every country globally, with millions infected and hundreds of thousands dead. More than 400,000 people died of the disease, and the mortality rate reached approximately 10% in some countries (e.g., Italy and Spain). Currently, the disease has been controlled in China, an achievement attributed to the dedication of all medical workers, as well as powerful and effective public health measures. However, medical workers worldwide are still fighting the disease. Public health measures are the most effective way to control the outbreak of an emerging infectious disease because effective drugs and vaccines will not be available in the coming months. However, the treatment and prevention of an infection depend on effective drugs and vaccines, particularly when COVID-19 becomes a common disease. In China and other countries worldwide, many clinical trials are being conducted in which drugs available in the market are screened for their efficacy in treating patients with COVID-19. The R&D of new drugs is ongoing as well. Simultaneously, more than one hundred institutions and companies are engaged in the development of COVID-19 vaccines. In addition to traditional inactivated vaccines, new technologies are being employed to develop COVID-19 vaccines, such as mRNA/DNA vaccines, genetically engineered vaccines, and vaccines based in adenovirus-based vectors. An mRNA COVID-19 vaccine developed by Moderna was first tested in humans in the United States of America on March 16. An Ad5-nCov vaccine phase 1 trial was initiated in China on the same day, 2 and promising results have been reported. 3 Until end of July, 2020, more than ten COVID-19 vaccines have entered the stage of human clinical trials. An increasing number of vaccines will initiate clinical trials to evaluate the efficacy and safety. Here, the challenges in the design of COVID-19 vaccine trials and evaluation of vaccine efficacy are discussed from a statistical perspective. The general framework of vaccine clinical development is first introduced in the following section. Due to the pandemic, the urgent demand for a vaccine brings distinct challenges to both exploratory and confirmatory studies of COVID-19 vaccines, which are discussed in Section 3. The possibility and feasibility of various designs, such as "individual-level" efficacy trials, stepped-wedge trials, and ring vaccination trials, are considered in the setting of confirmatory trials. Innovative statistical designs, including seamless design, platform trial, and master protocol design, are proposed to accelerate clinical development. The importance of a Data Safety Monitoring Board (DSMB) is also emphasized in this section. Section 4 provides concluding remarks. The clinical development of a new vaccine is divided into three phases before marketing. Phase I trials are dose-escalation studies used to evaluate safety. The vaccine is first tested in lowrisk populations and then in high-risk populations, such as adults transitioning into the elderly population. The trials stop once the protocol-specified termination criteria are met. Phase II trials are conducted to find optimal vaccination dosages and administration regimens after their safety has been evaluated. The immunogenicity endpoint is employed to evaluate efficacy as a surrogate to choose optimal dosages and administration regimens. Phase I and II trials are conducted to address the exploratory objectives of a new vaccine, including preliminary safety evaluation, dosages, and administration regimens. Phase III trials, also called confirmatory trials, are pivotal studies to evaluate the safety and efficacy of a vaccine. The "gold standard" to evaluate the efficacy of a vaccine is whether it can lower the incidence of the disease more than a placebo. However, efficacy trials require large sample sizes, long study periods, and high budgets. Immunogenicity trials can only replace efficacy trials if the relationship between immunogenicity and vaccine protection is well established. The immunogenicity endpoint is often accepted as a surrogate for efficacy, as was the case for rabies, varicella, and poliovirus vaccines. For a new vaccine such as the COVID-19 vaccine, an efficacy trial is unavoidable because immunogenicity trials cannot replace it. A potential alternative would be a challenge trial, which has been performed for the influenza virus and V. cholerae. However, this strategy will raise additional ethical concerns 4-6 and is not allowed in some countries. The primary objective of exploratory trials is to find an optimal dose and administration schedule. A global public health emergency makes the availability of an effective vaccine urgent. However, the lack of sufficient research and understanding about the disease and its causative virus is a major obstacle for the conduction of exploratory trials. In virtue of the pandemic, smaller dose numbers and shorter schedules than usual are preferred to achieve early protection in vaccinated subjects, especially when the emergency use of the vaccine is necessary. The lower doses also put lower pressure on manufacturing companies, which helps ensure that there is enough supply for the populations. Compared with a regular dose schedule, a rapid schedule with a shorter interval between two adjacent doses is proposed for emergency use as long as immunogenicity is sufficient. Moreover, immunogenicity persistence is another concern, since emergency schedules might not provide similar immunogenicity persistence compared to regular schedules. Here, a booster dose is required. However, all these possibilities depend on the results of exploratory studies. These might be different in various vaccines, and detailed discussions must be made case by case. A new vaccine must be evaluated rigorously and objectively before becoming available. As mentioned above, a field efficacy trial is inevitable to evaluate the efficacy of a new vaccine, which is also proposed by the World Health Organization (WHO). 7 COVID-19 will likely become a recurrent epidemic with seasonal outbreaks. Furthermore, emergency vaccination in high-risk populations, such as the first-line medical workers, is necessary. It is possible for regulatory agencies to conditionally approve a vaccine for early and restricted use when immunogenicity and safety data are promising. However, it is challenging to find a quantitative relationship between immunogenicity and vaccine protection, as it requires considerable supportive epidemiological data. At this point, it is impossible to build such a correlation for the COVID-19 vaccine. The immunogenicity data from convalescent plasma might be used as a reference for the optimal dose selection and to assess whether post-vaccination antibody levels are high enough to provide protection. 8, 9 Alternatively, cell-mediated immunity (CMI) is also a measure of immunogenicity. Both humoral and cellular immune responses contribute to vaccine protection. The proposed solution is a mathematical model to predict vaccine protection from both antibody and CMI data, which leads to a more accurate estimate of vaccine efficacy. 10, 11 However, previous studies aimed to model vaccine protection only from the antibody levels. 11, 12 The prediction model from both antibody and CMI data is more complicated and requires further studies. In an efficacy trial, randomization is usually performed on a subject basis and is named an "individual-level efficacy trial" hereafter. It evaluates the efficacy of vaccines and should be considered before making the vaccine available. The "clusterlevel efficacy trial," in which randomization is conducted in clusters, is common in real-world vaccine studies. It usually takes a community or village as a unit. Compared with the individuallevel efficacy trials, the cluster-level efficacy trials evaluate the vaccine's effectiveness in a real-world environment and have better-operating characteristics. However, they originate more confounding factors as well. For example, bias from covariates at baseline might be introduced due to cluster randomization, which cannot guarantee the balance of covariates within a cluster. Therefore, cluster-level efficacy trials cannot replace individuallevel trials to evaluate vaccine efficacy. Stepped-wedge trials 13, 14 and ring vaccination trials 15 are two typical cluster randomization designs in vaccine studies. Compared with a parallel cluster randomization trial, the stepped-wedge trial is a type of one-direction cross-over design. 13 No cluster receives the intervention of interest at the first time point. The clusters are randomized to initiate the intervention at subsequent time points, and the response to the intervention is measured. The ring vaccination trial is designed to recruit subjects who are socially or geographically connected to a case and, therefore, at increased risk of infection and developing disease within a few weeks. 15 The phase 3 trial of Ebola vaccine set an example of ring vaccination study during the outbreak in Guinea. 15, 16 It is particularly used in situations evolving rapidly. In a ring vaccination trial, people in contact with one case are identified as a ring and randomized to take either a vaccine or a placebo. The detection of new cases drives the recruitment of ring vaccination trials. The people in contact with the cases have higher probabilities of developing the disease, leading to smaller sample sizes and shorter study durations compared to individual-level trials. Moreover, in the Ebola efficacy trial, the control group subjects took a delayed vaccination due to ethical considerations. The delay period of 21 days after vaccination in treatment groups allowed 95% of cases to raise and enabled the detection of vaccine efficacy. It seems that ring vaccination trials offer a good option to evaluate COVID-19 vaccine efficacy during the outbreak. However, the operational characteristics are challenging. The detection of a new case and the identification of the patient's contacts require the input of local public health authorities. The rapid detection and identification of contacts are directly related to trial conduction and efficacy evaluation. If the people in contact with the patients are not found and vaccinated on time, vaccine efficacy might be underestimated because late vaccination might not stop disease progression. Furthermore, the feasibility of ring vaccination trials depends on the dose schedule and virus incubation periods. If one dose is enough to prevent COVID-19, ring vaccination trials may be feasible. The cases from the treatment and control groups are collected after vaccination to evaluate vaccine efficacy, and delayed vaccinations are then grouped in the placebo arm due to ethical concerns. However, vaccine protection cannot work immediately after vaccination. The cases occurring immediately after vaccination may be caused because vaccine protection is not yet achieved. The chance to underestimate the vaccine's efficacy is larger when the incubation period of the disease is shorter than the time needed for the vaccine to show its effects. Furthermore, more problems arise if two or more doses are needed to produce enough protection. It is impossible to collect cases after all doses are administered if the interval between two doses is longer than the 14-day incubation period of COVID-19. Ring vaccination trials cannot be performed to evaluate vaccine efficacy after two or more doses of immunization. Alternatively, the subjects in the placebo arm cannot wait long for delayed vaccination after exposure, as it does not meet ethical requirements. However, ring vaccination trials are currently the best solution to evaluate vaccine efficacy during the outbreak of a disease and help prevent public health emergencies. If there is no outbreak, but COVID-19 is still prevalent, individuallevel efficacy trials are recommended for drug registration because they produce more accurate vaccine efficacy estimates than cluster-level trials. However, when public health emergencies occur, trials not only contribute to evaluating vaccine efficacy but also prevent the spread of the disease. During a public health emergency, cluster randomization trials are an option, even though ring vaccination trials cannot be conducted in multi-dose vaccinations. The rapid development of a COVID-19 vaccine is expected to lead to the protection of humans from the disease. The seamless design is the first choice to accelerate the development at the clinical stage, as was the case of phase I/II trivalent influenza virus vaccine trials, 17 and phase IIb/III human papillomavirus (HPV) vaccine trials. 18 Phase I/II seamless trials combine dose-escalation stages for preliminary safety evaluation (phase I stage) and dose/schedule finding (phase II stage) into one study. The subjects from phase I stage are combined with the ones from the phase II stage to help to find the optimal dose and schedule. The total sample size is decreased, and the duration of the study is shortened. Phase II/III seamless trials are another way to shorten clinical development while including the exploratory (phase II) and confirmatory stages (phase III). They are pivotal studies to confirm the vaccine efficacy and safety, providing crucial evidence for registration. As a result, more challenges must be overcome from statistical perspectives. In a seamless phase II/III trial, sample size re-estimation based on unblinded interim results from the first stage is considered, which brings the inflation of the familywise type I error. The treatment arm, which has inferior efficacy results based on interim analysis, might be dropped from the trial (referred to as "drop-the-loser arm"), and only a selected dose is administered at the second stage. Furthermore, when interim results from the first stage show promising efficacy, fewer subjects might be proposed to randomize to the placebo arm, which is always desirable due to ethical considerations. Then, the allocation ratio is adapted at the second stage. Both "drop-the-loser" arm and allocation ratio adaptation have the risks of undermining trial integrity and lead to a biased estimate of vaccine efficacy. 19 The measures to control familywise type I error and statistical methods leading to correct estimates must be pre-defined in the protocol. A comprehensive simulation is highly recommended to evaluate the potential risks and benefits of adaptations in phase II/III trials. The choice of optimal doses and schedules should be made by an Independent Data Monitoring Committee (IDMC) to maintain the blindness of the trial. The interim results are reviewed by IDMC but not by sponsors and investigators. A reasonable and executable interim plan should be pre-specified in the protocol. If no dose-response relationship from a limited sample size at the first stage is explicit, it is hard for an IDMC to choose optimal doses according to the pre-specified interim plan. The trial must be terminated, and another exploratory study is required. It has potential risk to dose and schedule finding in seamless phase II/III trial if preliminary data of the product from exploratory studies is not enough and interim plan pre-specified in the protocol is neither clear nor executable. A phase IIb/III trial to support the 9-valent HPV vaccine distribution is a successful example of a seamless design trial. 18 However, phase II/III seamless trials are not recommended if the candidate vaccine is not sufficiently investigated. Moreover, the complexity of phase II/III trial designs, including control of familywise type I error, maintaining blindness during the trial, and choosing appropriate statistical methods, is another challenge. Therefore, phase I/II seamless trials are preferred and recommended rather than phase II/III trials to develop a COVID-19 vaccine. The platform trial is another potential method to accelerate COVID-19 vaccine development. A platform trial is defined by the broad goal of finding the best treatment for a disease by simultaneously investigating multiple treatments. 20 In a platform trial, multiple treatments are evaluated at the same time and share one control group. It greatly improves the efficiency of clinical development and saves resources. 20, 21 The platform trial is especially appealing to a vaccine efficacy trial because a larger sample size is required. Multiple COVID-19 vaccines may merge into one platform trial to evaluate vaccine efficacy compared with one placebo. The Innovative Medicines Initiative of the European Union launched a Bayesian platform trial to prevent Alzheimer's disease. 20, 22 "Solidarity" was launched by the WHO to find an effective treatment for COVID-19. 23 However, the operational characteristics of a platform trial are challenging. Multiple vaccines included in one platform trial may be from different sponsors, which brings conflicts of interest and needs further negotiations. Different dose regimens of participating vaccines also bring difficulties to the trial's design. Because the COVID pandemic has a dynamic nature, it is risky to start a field efficacy trial to evaluate a vaccine's efficacy in a single country/region. A multi-regional clinical trial (MRCT) conducted in different countries and regions minimizes the risk. However, one of the challenges in MRCT is to follow an identical protocol in all locations, which may not satisfy the regulatory requirements of all participating countries and regions. To accelerate the conduction of a trial in multiple countries and regions, we propose a master protocol design. In oncology, the US FDA proposed a master protocol design to simultaneously evaluate more than one investigational drug and more than one cancer type within the same overall trial structure. 24 In this discussion, we suggest the application of the same concept to vaccine trials. In this design, a master protocol is first written. Then, a subprotocol is drafted from the master protocol for each participating country/region and modified according to the participating country/region's regulatory requirements. The subprotocols must be consistent with the master protocol in crucial aspects, such as diagnostic standards of new cases, the primary endpoint, and core inclusion/exclusion criteria. Compared with traditional MRCT, this design avoids the effect of various regulatory requirements on the master protocol, making it possible to start the trial in a given country/region quickly. DSMB is a third-party committee that monitors vaccine safety regularly during the process of clinical development to detect potential safety risks and protect the interests of participants in early stages. It is composed of clinicians, epidemiologists, and statisticians. The safety of a COVID-19 vaccine is completely unknown, and its rapid development cannot sacrifice safety concerns. Hundreds of millions of people worldwide may take the vaccine once it becomes available. It is a great challenge that a huge population is exposed to the vaccine in such a short period. In the limited sample size of premarketing trials, we must detect any safety signals to avoid post-marketing risk. Therefore, the DSMB becomes more critical in those cases. Moreover, when vaccine trials are conducted in different countries and regions, establishing a program-level DSMB that monitors all safety data within the program and helps detect potential safety risks of the product in all participating countries and regions is recommended. The pandemic of COVID-19 calls for the urgent and rapid development of a vaccine. A COVID-19 vaccine is a final solution to defeat this coronavirus completely. However, the efficacy and safety of the vaccine must be guaranteed through a series of clinical trials. Otherwise, such an intervention may be a disaster. Innovative trial designs, including ring vaccination trials, seamless design, platform trials, and master protocol design, can accelerate the process of clinical development. However, we face more challenges in this urgent situation. First, ring vaccination trials are currently the best way to evaluate vaccine efficacy during the outbreak of a disease and simultaneously prevent its spread. However, its use is restricted to a dose/schedule of the vaccine and the incubation period of the disease. A new trial design to overcome the disadvantages of ring vaccination trials is appealing. Second, the prediction model of vaccine protection from antibody and CMI data might help evaluate vaccine efficacy early. However, current statistical methodologies and research are insufficient, and more efforts are required. Finally, the seamless phase I/II design was applied in multiple COVID-19 vaccines. [25] [26] [27] However, the application of a seamless phase II/III trial should be done carefully because of the statistical issues brought by flexible adaptations. It is not proposed that exploratory data are not enough but a seamless phase II/III trial is expected. If a phase II/III trial is used, it must be rigorously designed. Coronavirus COVID-19 global cases by the center for systems science and engineering (CSSE) at Johns Hopkins University (JHU) COVID-19 vaccine candidates: 6 front-runners Safety, tolerability, and immunogenicity of a recombinant adenovirus type-5 vectored COVID-19 vaccine: a dose-escalation, open-label, non-randomised, first-in-human trial COVID-19 human challenge studies: ethical issues World Health Organization. Human challenge trials for vaccine development: regulatory considerations World Health Organization. Key criteria for the ethical acceptability of COVID-19 human challenge studies Novel coronavirus: COVID-19 phase IIb/III vaccine trial synopsis Specific neutralizing response in plasma from convalescent patients of ebola virus disease against the West Africa Makona variant of Ebola virus COVID-19 vaccines: neutralizing antibodies and the alum advantage Randomized, double-blind, phase 2a trial of falciparum malaria vaccines RTS,S/AS01B and RTS,S/AS02A in malaria-naive adults: safety, efficacy, and immunologic associates of protection World Health Organization. Correlates of vaccine-induced protection: methods and implications Correlates of protection induced by vaccination Design and analysis of stepped wedge cluster randomized trials The stepped wedge cluster randomized trial: rationale, design, analysis, and reporting Ebola ça suffit ring vaccination trial con,sortium. The ring vaccination trial: a novel cluster randomized controlled trial design to evaluate vaccine efficacy and effectiveness during outbreaks, with special reference to Ebola Efficacy and effectiveness of an rVSV-vectored vaccine in preventing Ebola virus disease: final results from the Guinea ring vaccination, open-label, cluster-randomized trial (Ebola Ça Suffit!) Phase I/II trial of a replication-deficient trivalent influenza virus vaccine lacking NS1 A seamless phase IIB/III adaptive outcome trial: design rationale and implementation challenges Adaptive designs for clinical trials of drug and biologics guidance for industry The platform trial: an efficient strategy for evaluating multiple treatments Efficiencies of platform trials: A vision of the future Re-engineering alzheimer clinical trials: global alzheimer's platform network Solidarity" clinical trial for COVID-19 treatments Master protocols: efficient clinical trial design strategies to expedite development of oncology drugs and biologics guidance for industry Study to describe the safety, tolerability, immunogenicity, and potential efficacy of RNA vaccine candidates against COVID-19 in healthy adults Phase Ib-II trial of dendritic cell vaccine to prevent COVID-19 in adults Phase I/II clinical trial of recombinant novel coronavirus vaccine (adenovirus type 5 vector) in Canada We would like to thank two anonymous reviewers and editors for their constructive comments.