key: cord-0730450-apkuzuvw authors: Meeker-O’Connell, Ann; Stewart, Ansalan; Glessner, Coleen title: Can quality management drive evidence generation? date: 2021-11-02 journal: Clin Trials DOI: 10.1177/17407745211034496 sha: 0c845ab685e212c723253a35575b3d18bfc4a90f doc_id: 730450 cord_uid: apkuzuvw Recent guidance documents from international regulators emphasize the importance of thoughtful trial design and risk-based oversight in delivering reliable results. In practice, these recommendations are often implemented in a fragmented manner, reducing their effectiveness. We argue that collaborative, cross-stakeholder engagement that prioritizes both optimal trial design and tailored oversight are a necessary and effective approach to modernize quality management. This practice is at the core of Quality by Design, an approach that involves identifying important errors that could undermine trial credibility or participant safety and addressing them proactively. While Quality by Design is well suited for clinical trials supporting regulatory approval of a new medicinal product, we describe how the approach is equally relevant for pragmatic trials, including those conducted in the context of a pandemic. The race to efficiently evaluate potential therapies for COVID-19 during the pandemic has underscored the need for modernized approaches to testing and evaluating the safety and effectiveness of medical products. Modernization must extend beyond regulatory science considerations to include reimagining the way clinical trials are designed and executed. The US Food and Drug Administration and other global regulatorsalong with organizations representing industry, trialists, and patients-have long urged sponsors to re-examine use of a traditional approach to clinical trial quality, most notably having clinical trial monitors and auditors verify the accurate transcription of each collected data point, regardless of the overall impact on the reliability of trial conclusions. 1 Instead, regulatory guidance increasingly emphasizes the importance of trial design. During design, the protocol can be modified to reduce the opportunity for important errors during study conduct, including the risk that trial complexity itself undermines successful completion. 2 As the French writer Antoine de Saint-Exupe´ry said, ''Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.'' To this end, the concept of Quality by Design for clinical trials has evolved. The phrase may sound bureaucratic, but the approach is simple. It involves translating insights from trial stakeholders into an optimized trial design, decreasing opportunity for important errors. Similar to the concept of ''important protocol deviations'' described in the International Council for Harmonisation (ICH) E3 Questions and Answers guideline, ''important errors'' are a subset of the deviations that may occur in the end-to-end delivery of a clinical trial. 3 These errors could meaningfully impact the safety of trial participants and/or the credibility of trial results (which ultimately inform patient care). For example, prospective dialogue among trial designers, the study team, clinicians, potential investigators, and patients may reveal that the primary endpoint definition requires refinement to be interpreted consistently across trial sites. Such dialogue could also reveal evolving knowledge about the natural history of the disease, like the manifestations and timing of clinical characteristics, which may lead to re-evaluation of a planned secondary endpoint. Consider an example where new diagnostic methods can detect clinically important diminution of function earlier than previously feasible, necessitating a change in the timing and method of assessment of an intervention's efficacy in preventing that deterioration. Understanding potential challenges early permits trial designers to refine the endpoint definitions and assessment approaches and to plan for these to be a focus of protocol training. Quality by Design also sets the stage for more riskinformed, tailored oversight, facilitating the modernized approaches to clinical trial monitoring promoted by international regulators. 4, 5 Rather than focusing equal attention on every activity or data point in the trial, the study team deploys tactics to minimize the potential for important errors in trial conduct, critical data capture, and reporting that could not be eliminated through trial design. For example, in an outcomes-driven trial, differential retention of participants across randomized arms may introduce bias, and operational plans may focus on full ascertainment of events, complete follow-up, and reducing the number of participants lost to followup. In this way, benefits of early planning extend into trial operations and oversight (whether by monitors or auditors), retaining focus on those activities that are essential to the credibility of the study outcomes and/or to participant safety (see Figure 1 ). Preventing important errors through trial design and focused oversight is a priority for regulators. This concept has been successfully applied by trialists and is incorporated in the ongoing renovation of the ICH Guidelines related to clinical trial design, planning, management, and conduct: ''E8 (R1) General Considerations for Clinical Trials'' and ''E6 (R2) Guideline for Good Clinical Practice.'' 4,6,7 The Both ICH E8 (R1) and ICH E6 (R2) stress an upfront investment in planning by trial sponsors, understanding that the cost of a miscalculation is nearly always higher than the cost of added time to plan and conduct the trial strategically, and that small investment of time prospectively may expedite trial completion and analysis. 8 In the ongoing renovation of ICH guidelines, this link between prospective planning (per E8) and Good Clinical Practice (per E6) may be made even more explicit. 9 Derived from the pioneering work of engineers, scientists, and statisticians who devised systematic approaches to quality improvement in the early 20th century, Quality by Design is not a novel concept for many industries. 10 The approach has been broadly adopted in the manufacturing sector since the 1970s to improve the quality and efficiency of producing goods ranging from electronics to pharmaceuticals. Trialists looking to improve trial design and conduct have had some recent success with in adapting the concept to medical product development, but broader adoption across the clinical trials enterprise is merited. 3, 4 To support effective implementation of Quality by Design, the Clinical Trials Transformation Initiative (CTTI), a public-private partnership co-founded by the Duke University and the US Food and Drug Administration, has developed a variety of online resources. 11 Improved adoption will also require both re-evaluation of how trials are designed, conducted, and overseen within organizations and willingness to acknowledge the opportunities that remain improve trial design. While the ICH guidelines emphasize interconnectedness, the practical implementation of these guidelines often occurs in silos within research organizations. Teams responsible for conducting, overseeing, and inspecting trials may focus on adhering to ICH E6, while trial designers and reviewers may focus on the concepts in ICH E8. This differential focus can result in duplicative work and missed opportunities to address important risks upfront. For example, those designing a trial may focus on whether the proposed primary endpoint is clinically relevant, likely to be impacted by the intervention, and appropriately timed for assessment. Conversely, those overseeing a trial may focus on the risk that a participant does not complete a required primary endpoint evaluation at the appropriate time or that its documentation is incomplete. Without effective communication, neither party may appreciate other key risks, such as differences in medical practice across participating centers leading to challenges in conducting the primary endpoint assessment at the protocolindicated frequency. Both trial designers and trial overseers are well intentioned but may not be successful in driving efficient, streamlined trial design, conduct, and oversight. Ongoing collaboration among stakeholders is key, including global and local regulators where relevant. Such collaboration-combined with a keen focus on only the critical data necessary to obtain robust results that could rapidly impact clinical care-allowed the RECOVERY trial of potential COVID-19 therapies to move from draft protocol to enrolling participants in 9 days. Moreover, real-time insights from hospital staff on the pressures they faced in managing COVID-19 patients prompted significant streamlining of eligibility criteria, procedures, and data collection to minimize additional burden in an environment already stressed by the pandemic. 12 The principles of Quality by Design also drove consideration of individual clinician judgment on treatment suitability for patients who might be eligible for this pragmatic trial, balancing the risks of the trial with the context of usual care. While RECOVERY may serve as an exemplar of the approach, many other trials initiated to address the COVID-19 pandemic have been uncontrolled, underpowered, and thus unlikely to yield conclusive results to guide patient care. 13 Even in routine settings and in trials conducted to support regulatory approval of a product, investigators and potential trial participants have valuable insights, ranging from the feasibility of trial assessments to the general relevance of study endpoints to the targeted patient population. When a study has novel design features that may influence evaluability (e.g. patient populations, procedures, or endpoints), early engagement with regulators may also inform trial design and oversight. With Quality by Design, the clinical trial community is recognizing the value of designing clinical studies inclusively from the beginning, with the scientific merits, the clinician and patient perspectives, and the operational feasibility all in mind. By carrying forward these principles and applying them broadly, the clinical trials enterprise can increasingly prevent important errors, better ensure collected data are fit for purpose, reduce participant burden, and, ultimately, advance public health efficiently and effectively into the future-a need that has never been more pronounced than in the face of the current pandemic. The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The funding for this work was made possible, in part, by the US Food and Drug Administration through grant R18FD005292 and cooperative agreement U19FD003800. The views expressed in written materials or publications and by speakers and moderators do not necessarily reflect the official policies of the Department of Health and Human Services, nor does any mention of trade names, commercial practices, or organization imply endorsement by the United States Government. Partial funding was also provided by pooled membership fees from Clinical Trials Transformation Initiative's member organizations. Guidance for Industry. ICH E8(R1) General considerations for clinical trials Enhancing clinical evidence by proactively building quality into clinical trials E3 Structure and content of clinical study reports: questions and answers (R1) E6(R2) Good clinical practice: integrated addendum to ICH E6(R1 Oversight of clinical investigations-a risk-based approach to monitoring The magic of randomization versus the myth of real-world evidence Randomised evaluation of COVID-19 therapy (RECOVERY Diagnosing protocol amendment experience to drive clinical trial performance International Council for Harmonisation. E6(R3) draft principles good clinical practice The history of quality How to set up a trial in nine days The authors recognize Dagmar Go¨rtz, Louise Bowman, and Karlin Schroeder for critical review of the manuscript, and Lisa Kepler for medical writing support. A.S. wishes to acknowledge Office of Medical Policy colleagues for formative discussions and critical review of an earlier draft.