key: cord-0011047-c1olcki4
authors: Ball, Greg; Kurek, Raffael; Hendrickson, Barbara A.; Buchanan, James; Wang, William W.; Duke, Susan P.; Bhattacharyya, Amit; Li, Mengchun; O’Brien, Dennis; Weigel, Judith; Wang, Wenquan; Jiang, Qi; Ahmad, Faiz; Seltzer, Jonathan H.; Herrero-Martinez, Esteban; Tremmel, Lothar
title: Global Regulatory Landscape for Aggregate Safety Assessments: Recent Developments and Future Directions
date: 2020-01-06
journal: Ther Innov Regul Sci
DOI: 10.1007/s43441-019-00076-4
sha: a6efa095bb56b8e7d79591cea86a74556e16f5e8
doc_id: 11047
cord_uid: c1olcki4

Notwithstanding successful harmonization efforts, the global regulatory framework governing product safety is complex and continually evolving, as evidenced by additional regional guidance and regulations. In this regulatory review, we provide an overview from both global and regional perspectives. A historical perspective, with a focus on recent developments, enables identification of important long–term trends, such as a shift from single–case medical review of serious adverse events to an interdisciplinary evaluation of aggregate data for the purpose of judging product causality and informing benefit–risk assessments. We will show how these trends lead to opportunities for closer interdisciplinary collaboration, for bridging the gap between preand postmarketing surveillance, and for a more proactive determination of patient populations with a positive benefit–risk profile for product use. We will conclude by pointing to ongoing and future work that seeks to provide specific solutions for ongoing aggregate safety evaluation.

Throughout the life cycle of a medicinal product, interdisciplinary safety evaluation is needed to understand the evolving safety profile and to ensure effective risk-management strategies. Three overlapping stages of safety evaluation are generally considered during the course of clinical development: clinical trial safety monitoring, safety specification (principally at submission for a marketing authorization), and postmarketing safety surveillance (generally more intensive early in postauthorization). ICH 1 guidance forms an evolving framework for medicine life cycle management, including for assessment of patient safety during product development. Globally, although there are divergence and delays in implementation and varying modifications in subsequent legislation, key stakeholders, including sponsors, regulatory authorities, and academics, generally accept this guidance. Since the 1980s, CIOMS, 2 an international, cross-functional, nongovernmental organization, has been shaping, with its expert working groups, the thinking and methodological approach for pharmacovigilance and influencing the development of corresponding ICH guidelines. Emphasis on an interdisciplinary process and the need for analysis of safety based on aggregate data, rather than only medical review of individual cases, have been recurring central concepts.

Regional regulatory initiatives can also trigger an evolution of the existing framework. Recent guidance 3,4 on safety monitoring during medicine development, issued by regulatory authorities in the United States and European Union, indicate a shift in focus toward aggregate safety monitoring and scientific evaluation of integrated safety data earlier in the development program. Early identification of safety risks enables termination of programs when risks are not supported by anticipated benefits for the indicated patient population. Alternatively, improved characterization of the safety profile of medicinal products could lead to identification of risk mitigation measures that would avoid termination of programs showing promise even in the face of recognized risks.

This shift in emphasis provides an opportunity for stakeholders to engage in the advancement of crossdisciplinary procedures for aggregate safety analysis to further improve identification and characterization of risks for a medicine on a program level, especially at the stage of clinical trial safety monitoring. This work requires interdisciplinary expertise, enhanced processes and innovative methodology. Success will require dynamic, interactive collaboration between clinical and statistical scientists. Clinical scientists in this context refer broadly to clinical and safety physicians, nurses, pharmacists, and other medical experts; similarly, statistical scientists refer to statisticians, epidemiologists, and other quantitative scientistsall involved in medicine development. The joint Drug Information Association-American Statistical Association (DIA-ASA) Interdisciplinary Safety Evaluation scientific working group has recognized this opportunity and is developing specific deliverables to better enable quantification and analysis in safety monitoring during clinical development, safety specification at marketing authorization submission, and safety surveillance once medicines are on the market.

The primary goal of our working group is to empower the broader cross-disciplinary, cross-regional community to develop and promote practical quantitative solutions for safety evaluation during clinical development. This paper reviews the current regulatory landscape with a focus on the concept of aggregate safety analysis, considering perspectives from CIOMS, ICH, the US, and EU. We identify similarities and differences in current thinking. In addition, we discuss procedural and methodological requirements needed to further advance this field. In a continuing series of articles, we will build on this foundation, providing specific and actionable solutions for aggregate safety assessment planning and ongoing aggregate safety evaluation.

CIOMS (founded in 1949 by WHO and UNESCO) is an important strategic public health organization for advancing international pharmaceutical product development and pharmacovigilance (PV) practices, influencing and persuading through the strength of the ideas and credibility of its expert working groups. CIOMS reports on PV (listed in Table 1 ) have influenced guidance from ICH and regulatory authorities, including the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA). Three of these reports, which we will discuss, align with the 3 fundamental, overlapping stages of safety evaluation (safety monitoring, safety specification, and safety surveillance).

In 1986, CIOMS convened its first PV working group 5 with the purpose of making recommendations on the standardization of individual case safety reports by sponsors to regulatory authorities. The working group developed standardized definitions, procedures, and format, including the CIOMS I reporting form which, for the first time, set the minimum standard for safety reporting and is the basis for many current national adverse drug reaction (ADR) reporting forms. CIOMS I has directly influenced technical requirements for ICH, such as ICH E2A (discussed in section 2.2.1), which covers expedited reporting in clinical trials.

CIOMS VI: Management of Safety Information from Clinical Trials CIOMS VI 6 aspires to bridge the gap between pre-and postapproval activities for the understanding and management of risks. It discusses the importance of a systematic approach for managing risks during development, with a special focus on clinical trials. As the safety database matures, aggregate analysis becomes ever more important than just individual case review for detection and evaluation of signals. The core idea of CIOMS VI is implementation of productspecific, multidisciplinary safety management teams for regularly reviewing program-level safety information (from ongoing blinded trials, completed unblinded trials, and other data sources), making decisions on safety and risk management in a more timely and informed manner, ensuring earlier identification of safety concerns and taking appropriate risk mitigation steps. The principles of this systematic approach are also the basis of the Program Safety Analysis Plan (PSAP), 7 recommended by the Pharmaceutical Research and Manufacturers of America (PhRMA) Safety Planning, Evaluation and Reporting Team (SPERT). The PSAP supports the planning and preparation for a systematic scientific evaluation of integrated safety information for new product submissions and during early postmarketing.

CIOMS VIII: Signal Detection in Pharmacovigilance CIOMS VIII 8 proposes quantitative methods for postmarketing signal detection using PV databases. This document presents a discussion of challenges in analyzing and interpreting spontaneous data. Signal detection, signal management, evaluation strategies, and stakeholder perspectives are also discussed.

ICH was formed in 1990 with the overarching goal of standardizing the global drug registration and approval process. Both regulatory and industry members serve as partners in developing a common understanding of scientific and technical requirements needed for ensuring the safety and efficacy of pharmaceutical products for human use (the latest members and observers are listed here 9 ). Table 2 lists key ICH guidance on safety evaluation. This paper focuses on the guidance documents that allude to the need for more systematic aggregate safety evaluation during the 3 fundamental stages of safety evaluation: (1) clinical trial safety monitoring (E2A), (2) safety specification (and benefit-risk assessment) at submission (M4E), and (3) postmarketing safety surveillance (E2E).

ICH E2A: Clinical Safety Data Management: Definitions and Standards for Expedited Reporting ICH E2A, 10 which provides guidance on clinical safety data management, makes the important distinction between "adverse events" and "adverse drug reactions": while an adverse event does not necessarily have to have a causal relationship with the medicinal product, an ADR has to have facts or arguments to suggest at least a "reasonable possibility" of a causal relationship. Either the investigator or the sponsor can judge reasonable possibility. Consistent with this, European regulators 11 require that all reports that are unexpected per the Reference Safety Information (RSI) and considered related by either the investigator or the sponsor are to be reported as a 7/15 day report. In contrast, from the FDA's standpoint, 12 the sponsor has visibility to the totality of a product's safety data and is in the best position to judge causality; therefore, only sponsor judgment is to be used. A TransCelerate survey 13 has revealed that most companies are addressing the FDA IND final rule through medical review of clinical trial serious adverse events as part of the company causality assessment. Member companies have seen a substantial decrease in IND safety reports.

An ADR is "unexpected" if its nature or severity is not consistent with the RSI, based on clinical safety data previously observed, recognized, and documented (not on the basis of what might be expected from the pharmacologic properties of a drug). For an investigational drug not yet approved, the RSI is listed in the Investigator's Brochure. For approved medicines, this information can be found in the US package insert (USPI), EU Summary of Product Characteristics (SmPC) or Investigator's Brochure. ICH E2A states that a clinically ICH M4E (R2), which was revised in 2016 with a focus on the Benefits and Risks Conclusions (section 2.5.6), offers guidance on analyses of safety data at the clinical trial level and integrated analyses to evaluate benefit-risk at the product level.

Evaluation of drug safety continues throughout the product life cycle; as ICH E2A states, pre-and postmarketing safety monitoring concepts and practices are interdependent. This is aligned with CIOMS VI, which talks about bridging the gap between pre-and postapproval activities for understanding the safety profile of new medicines. Continued vigilance is essential because of postmarketing drug exposure in larger and more diverse patient populations compared to patients treated during controlled clinical trials.

ICH E2E 15 states that a safety specification plan and a PV plan should be built during the premarketing phase when approval is sought. The safety specification plan should summarize the identified and potential risks observed during clinical trials, along with any important missing information, and any actions designed to address these concerns. Signals from nonclinical studies that had not been adequately addressed as well as other limitations of the clinical safety database should be considered. Unresolved safety questions from studies that need further investigation should be noted. Experience from worldwide exposure of the drug class, epidemiology of the disease, and corresponding regulatory actions should also be listed. Ongoing aggregate safety evaluation of the accumulating data from postmarketing studies and clinical trials for potential new indications continues to inform postmarketing safety surveillance and the assessment of new signals.

This section focuses on regulatory authority-specific safety initiatives, which can indicate trends for future development of the regulatory landscape and provide opportunities for improving the existing framework.

In the US the emphasis has been on aggregate analysis and clinical trial safety monitoring leading to the safety specification at submission. See Table 3 for key FDA guidance documents on clinical safety evaluation. We will discuss 4 of these (identified with an asterisk), the earliest dating back to 1988. 

The 1988 FDA Guideline 16 for the Format and Content of the Clinical and Statistical Sections of an Application visibly hails from a different era, as evidenced by its distinct font that was commonly used on typewriters. One purpose of this guideline has been to promote closer cooperation of clinical and statistical functions by establishing the "full integrated clinical and statistical report" for a single study. In addition, the guidance contains sections with advice for the integrated safety assessments required for a new drug application. It would be hard to overstate the influence of this pre-ICH document on the community of clinical statisticians and investigators, both in the US and in other countries, seeking to gain marketing approval for medicinal products in the US. The safety-related section firmly establishes the Integrated Summary of Safety as the clinical safety centerpiece of a New Drug Application (NDA). The document calls attention to the advantages of pooled analyses. Grouping studies can give more precise risk estimates than analysis of single studies and can allow for examination of differences among population subsets and serious adverse events too rare to be detected in single studies. It clarifies, however, that an integrated summary is an overall analysis, examining all relevant studies together. Moreover, the document fails to discuss issues of using aggregate summaries without accounting for potential study differences, which could lead to misleading results when study and treatment are confounded. 17 individual studies, without a thoughtful integration of the overall safety experience, is not adequate for a safety review." Clinical reviewers are encouraged to collaborate with their biostatistician colleagues on the exploration and estimation of event rates, both for individual studies and pooled data. The guidance also acknowledges that approaches to evaluate safety differ substantially from those for efficacy.

Assessments of drug relatedness is not a primary purpose of investigators and this guidance states that investigator assessments are of limited use, because they are "highly dependent on information about the side effect profile of the drug available at the time of the study (eg, what is in the Investigator's Brochure), and are not informed by awareness of the entire safety database." This presages the emphasis on aggregate reviews of safety data in the Investigational New Drug (IND) Safety Reporting final rule 19 for judging "reasonable possibility" of a causal association between an event and study drug.

The Guidance on Pre-Marketing Risk Assessment 20 (also issued in 2005) further develops some important themes from the reviewer's guidance. It calls for examining the data quality and, in particular, the quality and consistency of the Medical Dictionary for Regulatory Activities (MedDRA) 21 coding of adverse events. Sponsors seeking approval must provide, from clinical trials, a body of evidence that properly characterizes the product's safety profile, based on patients who adequately represent the expected target population. In particular, common adverse events related to therapy and signals for serious and/or less common adverse events need to be identified, and pooled analyses should be carried out to more reliably estimate the risks.

The sponsor's ongoing aggregate evaluation of safety data during clinical development is a dynamic process that serves to protect human patients participating in clinical trials and to gain an understanding of the safety profile of the drug as early in its development as possible. The 2010 FDA IND Safety Reporting final rule and the 2012 final guidance 22 (FDA Safety Reporting Requirements for INDs) highlight the importance of aggregate analyses for determining "reasonable possibility" of a causal association with study drug for certain serious adverse events. These documents describe 3 types of situations that would require expedited reporting: (A) a single occurrence of an uncommon event that is often associated with drugs (for example hepatic injury), (B) a few occurrences of an event that is uncommon in the population exposed (for example tendon rupture), and (C) for events that are not informative as a single event, such as events expected with the investigational agent or those anticipated in the population independent of exposure to the investigational agent: for which an aggregate analysis is required to detect whether there is a higher-than-expected event frequency in the population receiving the intervention.

Type A and B events can be assessed quite well with traditional medical monitoring and medical safety review of individual case reports and case series. For type C events, in order to comply with requirements for aggregate IND safety reports and to improve the overall quality of safety reporting, sponsors need to have a systematic approach for safety evaluation based on data in the aggregate. In particular, sponsors should periodically review accumulating safety data collected across multiple studies (completed and ongoing), analyze the data in the aggregate, and make a judgment about the likelihood that the drug caused any serious adverse events. In accordance with the guidance, causality judgments based on aggregate analyses are more meaningful in cases of (1) a higher incidence of an anticipated event with study drug compared to control or (2) an increased frequency of a previously recognized adverse reaction. A serious adverse reaction would need to be reported if an aggregate analysis indicated that a specific event occurred more frequently in the drug treatment group than in either a concurrent or historical control group.

The FDA final rule and 2012 guidance firmly put responsibility for determining "reasonable possibility" of a drug-event association on the sponsor rather than the investigator, and acknowledge that this deviates from ICH E2A, where a judgment about association could be based on either assessment. On the other hand, consistent with ICH, either the investigator or sponsor can decide if an event is "serious." The guidance also describes "anticipated events," which are not included in the RSI (and therefore technically "unexpected") but that are commonly observed in the target population. The protocol or safety surveillance plan (described below) should specify the anticipated events; they would become reportable if an aggregate analysis indicated an increased frequency over what had been anticipated.

The FDA Safety Assessment for IND Safety Reporting (2015) 3 is the second of the two guidance documents that the FDA has issued to further clarify its final rule on IND safety reporting and emphasize the increasing importance of the sponsor's aggregate safety assessment. This draft guidance focuses on reporting requirements for serious adverse events that are only interpretable as suspected ADRs with aggregate analysis, not when observed as single events or small clusters. Notably, this guidance introduces two new concepts for conducting such analyses, which are not mentioned in the final rule: safety assessment committees and safety surveillance plans. The safety assessment committee is an interdisciplinary, programlevel committee charged with reviewing, on an ongoing and regular basis, accumulating serious adverse events and related safety data for a product in clinical development, collected across multiple studies (completed and ongoing) and from other sources. The safety surveillance plan prespecifies the composition and operating model of the safety assessment committee and identifies anticipated serious adverse events and how they would be monitored by the committee.

Aggregate analysis focuses on "anticipated serious adverse events" and "previously recognized serious adverse reactions" (which are included in the RSI but still need to be monitored for increased frequency). Based on accumulating results, sponsors must make judgments about the triggers for IND safety reporting. 23 Despite additional challenges, the final rule creates an opportunity to help drug developers streamline safety reporting requirements for the FDA so that they only need to report to the agency, in an expedited manner, unexpected serious adverse events that are reasonably likely to be related to study drug.

While emphasis of US safety regulations have been in the premarketing arena, the FDA has recently implemented the Sentinel initiative, 24,25 a national electronic system designed to complement the agency's existing adverse event reporting system (FAERS). 26 Through Sentinel, the FDA can rapidly access information from large electronic health care data sets (including electronic health records, insurance claims data, and registries) to monitor the safety of marketed medicinal products. A pilot to test the feasibility of and to develop the scientific approaches needed for Sentinel was completed in February of 2016. Safety monitoring activities in Sentinel continue to be refined and expanded, transforming data into evidence. 27 The European Union

Chapter II of the European Commission Volume 10 Clinical Trials Guidelines 28 covers safety reporting. It contains a Commission Guidance on suspected unexpected serious adverse reaction (SUSAR) reporting (CT-3) 11 and the EU ICH E2F 29 guidance for development of the Development Safety Update Report. The underlying CT Directive (2001/20/EC) 30 will be replaced by the EU CT Regulation, 31 expected to go live in early 2020. As much of the CT-3 content is contained in the Regulation itself, CT-3 guidance not included in the EU CT Regulation and its annexes will be converted to a set of Q&As that will come into effect at the same time as the regulation. Though there are some important regional variations, as with ICH E2E Pharmacovigilance Planning, all ICH guidelines have been adopted in the EU.

In the EU, investigators need to report all serious adverse events to the sponsor immediately, with the exception of those identified in the protocol or the Investigator's Brochure. Aggregate assessment is needed for the reporting of safety data. Sponsors are required to continuously weigh the benefits and risks in clinical trials, which necessitates ongoing safety evaluation and reporting of SUSARs to the concerned national competent authorities (directly or through the EudraVigilance CT Module). Ethics committees are also informed, although requirements vary among member states.

In both the EU and US, the sponsor describes, in the RSI, 4,32 the expectedness of an adverse reaction from the perspective of events previously observed, not on the basis of what might be anticipated from the pharmacologic properties of a medicinal product. The EU document, in particular, has provided significant additional clarification on the development and use of RSI. Expected serious adverse reactions should be restricted to serious adverse reactions that were previously observed and where, after a thorough assessment by the sponsor, reasonable evidence of a causal relationship between the event and the investigational medicine exists. The RSI should not include nonserious ADRs, though they should be listed elsewhere in the Investigator's Brochure. Assessments should be based on the comparative incidence of suspected serious adverse reactions in all previous and ongoing trials. Exceptions from the aggregate approach require a very strong plausibility of a causal relationship with the medicinal product and a robust justification based on medical judgment. No further considerations regarding statistical methods for aggregate analyses are provided, but this methodology by necessity underpins the generation of the RSI for the EU.

The legislation bolstering postmarketing PV (Directive 2010/ 84/EU 33 and Regulation [EU] No 1235/2010) 34 is implemented by the EMA/HMA Good Pharmacovigilance Practice (GVP) guidance and associated population-specific guidance. 35 

Module where aggregate safety assessment is described and plays an important role.

Module IX 36 describes MAH and member state signal detection from individual case safety reports in the Eudravigilance database. EudraVigilance is the system for managing and analyzing information on suspected adverse reactions to medicines that have been authorized or are being studied in clinical trials in the European economic area. The EMA operates the system on behalf of the EU medicines regulatory network, which also includes regulators from the member states. The process covers signal detection (looking for and/or identifying signals using data from any source, combining review of individual case safety reports and statistical analysis), signal validation (determining if data supporting a detected signal justify further analysis), and signal confirmation. Confirmed signals will be prioritized and assessed by the PV Risk Assessment Committee (PRAC), the EMA committee responsible for assessing all aspects of risk-management for human medicines. A one-year pilot was started in February 2018 to determine the optimum way to perform signal detection for EU medicinal products using EudraVigilance.

EU regulatory authorities have been using the EudraVigilance data analysis system, since it was first implemented, for performing signal detection. In November of 2017, MAHs were given system access for signal detection as part of an enhancement to EudraVigilance functionality that also includes central reporting for postmarketing ADRs. System access for signal detection is governed by a Revision to GVP Module IX, which requires MAHs to perform signal detection based on disproportionality outputs from EudraVigilance. Previously, MAHs had no access to EudraVigilance data and had to run signal detection exercises in other postmarketing databases or their internal safety database.

Two associated guidance documents, published in 2006 37 and 2016, 38 provide detailed information on statistical methods for signal detection with the EudraVigilance data analysis system. The updated version from 2016, which supersedes the initial 2006 guidance, aims to give a discursive account of the potential use of statistical disproportionality methods based on evidence from research activities. The document states that this guidance should be viewed as a scientific discussion of evidence-based methods, not as providing regulatory requirements. Notably, the guidance emphasizes the importance of integrating several different methods for a comprehensive and efficient signal detection system.

The EU has two legislative frameworks for pre-and postmarketing pharmacovigilance, and regulators work across the separate systems. The PRAC, for example, is involved in the review and approval of both pre-and postmarketing Risk Management Plans (RMPs), and the clinical trial legislation (both current and future) considers situations where clinical trials are in marketed products, linking in to the appropriate sections of the postmarketing legislation. Furthermore, aggregate analysis throughout the medicine's life cycle relies on different sources, including data from clinical trials. Where patient safety is concerned, the PRAC will also be involved in the review and analysis of data. Despite the legislative separation between premarketing safety monitoring and postmarketing safety surveillance, in practice, this activity is moving toward a continuum of interdisciplinary safety evaluation.

The Pharmaceuticals and Medical Devices Agency (PMDA) is Japan's independent administrative organization for overseeing all consultation and review work from the preclinical stage through approval and into postmarketing surveillance. For safety monitoring, sponsors are required to summarize the important risks, potential risks, and missing safety information of investigational new drugs as part of GVP. 39 In addition, good postmarketing study practice assures proper implementation of postmarketing surveillance and reliability of safety data obtained after marketing. The PMDA and the Ministry of Health, Labor and Welfare (a cabinet level ministry of the Japanese government) were founding regulatory members of ICH. Japan follows ICH technical requirements to evaluate the long-term safety of pharmaceutical products and to accommodate multiregional clinical trial data. However, data obtained from other regions must meet Japan's regulatory requirements and demonstrate that the data can be extrapolated to Japan's population. The PMDA requires confirmation that the recommended dosing regimen does not introduce any additional safety concerns for the Japanese population. In accordance with ICH E2A, sponsors must report ADRs within a specified time frame. Also, as part of early postmarketing phase vigilance, sponsors are required to collect adverse events from health clinics and hospitals for the first 6 months after marketing approval and to report relevant safety information to health care professionals. In addition, 6-10 years after approval, sponsors must re-evaluate and confirm efficacy and safety of marketed drugs. There are some differences compared with other regions; however, safety guidelines in Japan are mainly based on ICH technical requirements. Moreover, germane to this publication, the PMDA has not promulgated any specific guidance regarding ongoing aggregate safety assessment and reporting.

ADRs during postmarketing are reported and captured though the Japan Adverse Drug Event Report (JADER) 40 system, which is similar to FAERS. PMDA uses the JADER database for signal detection and to investigate possible associations between events and drugs. The Medical Information for Risk Assessment Initiative (MIHARI), 41 framed in 2014, is PMDA's initiative to use electronic health care data, in addition to spontaneous adverse event reporting, to evaluate postmarketing safety risks.

Over the past 30 years, the China National Medical Products Administration (NMPA), formerly known as the China Food and Drug Administration (CFDA), has been establishing a comprehensive PV system. The Center of Drug Evaluation and the Center for Drug Re-evaluation, under the China NMPA, handle premarketing and postmarketing drug evaluation, respectively. For premarketing safety monitoring, the Provisions for Drug Registration (2011) 42 articulate sample size requirements for drug applications. The provisions also specify, for international multicenter clinical trials, that applicants must report, to the China NMPA, any observed serious or unexpected adverse reactions associated with study drug in any country. The Center of Drug Evaluation issued draft guidance for postmarketing research studies (2013), 43 with the primary aim of supplementing important efficacy and safety information after approval.

The most dramatic changes to the regulatory environment in China have come in the last 3 years. Since 2015, the China NMPA has issued many new regulatory policies 44 and guidance 45 to accelerate the review and approval of new clinical trial applications and new drug applications and to improve the quality of new medicines and medical devices. PV requirements have become largely aligned with ICH principles and practices. In June 2017, the China NMPA joined ICH as a formal regulatory member. In January of 2018, it started implementing a suite of guidance 46 for PV to comply with ICH E2A, E2B (R3), and M1. With the adoption of these guidelines, only SUSARs as defined in ICH E2A (instead of all serious adverse events) will need to be reported to the Center of Drug Evaluation in an expedited manner; individual case safety reports will be reported using E2B (R3) format (instead of line listings); and MedDRA will be used as the standard dictionary. These changes, substantially advancing and broadening the global harmonization of safety monitoring and reporting, will have a major impact on the efficiency of PV practices for clinical development. However, as with Japan, China has not provided any specific guidance regarding ongoing aggregate safety assessment and reporting.

See Figure 1 for a timeline of key global developments in the evolution of guidance for ongoing aggregate safety evaluation during clinical development and early postmarketing.

The global regulatory landscape continues to evolve as regulatory authorities, sponsors and academics work together to advance and improve the processes and methods for evaluating aggregate safety data. In recent years, progressively more legislative documents have required aggregate safety evaluations to ensure that sponsors are adequately assessing the safety and benefit-risk profile of medicinal products throughout their life cycle (see the timeline). These changes have led to an intensifying need for sponsors to set up cross-functional teams for conducting ongoing program-level safety assessments. The concept and use of aggregate analysis have been actively explored and developed for more than 25 years. Nevertheless, substantial differences remain across regions in the focus and level of detail describing methods and principles.

CIOMS has been driving some of the new thinking about aggregate safety assessments, both in terms of clinical trial safety monitoring (eg, CIOMS VI, 6 VII, 47 and X 48 ) and of postmarketing safety surveillance (eg, CIOMS II, 49 IV, 50 and VIII 8 ), which have been reflected in ICH guidance. ICH has emphasized the evaluation of safety data in clinical study reports, submission packages, and periodic reports during development and into postauthorization, for example; ICH E2C (Periodic Benefit-Risk Evaluation Report), 51 E2E (PV Plan), 15 E2F (Development Safety Update Report), 52 and M4E (Common Technical Document). 14 However, ICH guidelines provide only high-level requirements, mostly focusing on descriptions and summaries of safety data.

Recently the FDA 3,22 and EU regulators 4 have strengthened requirements for aggregate analysis throughout development, especially regarding the reporting of SUSARs. SUSARs are considered meaningful safety information and they must be provided to regulators and investigators in an expedited manner. The common goal has been to improve the management and mitigation of risks by early detection of potential safety issues. In addition to existing regulations, regulators in both regions have issued additional guidance that lay out in greater detail expectations for a more systematic approach to identify and evaluate important safety information during clinical development. Despite some differences in thinking between the agencies, particularly in regard to expedited reporting of safety data from clinical trials, both agencies require sponsors to regularly conduct program-level reviews, considering data from completed and ongoing clinical trials.

In 2017, the Clinical Trial Facilitation Group (CTFG), issued an important additional guidance, the Reference Safety Information Q&A, 4 to augment the current EU clinical trial safety reporting guidance, 11 which will be updated in 2020 with the implementation of the EU CT Regulation. The CTFG, which represents the EU Member States on clinical trials, is a subgroup of the Heads of Medicines Agencies (HMA). The RSI lists expected serious adverse reactions, based on aggregate assessment, and forms the basis for the expectedness assessment of all "suspected" serious adverse reactions that occur in clinical trials and for periodic safety reporting. The CTFG document puts a high emphasis on the development, format, and use of the RSI; the sponsor is responsible for promptly reviewing all information relevant to the safety of the medicinal product and considering the significance of a report within the context of other reports. Only serious adverse reactions with a "reasonable causal relationship" should be included in the RSI. Aggregate safety analysis, although its principles are described on a high level as a requirement for the determination of expected serious adverse events, is not a specific discussion topic in the CTFG guidance and no consideration regarding procedures (methodologies or processes) is made.

In contrast, the FDA's IND safety reporting guidance 3,22 documents, while not as focused on the RSI, are more comprehensive with regard to aggregate assessment. Like the CTFG document, they provide guidance on the reporting of serious and unexpected adverse reactions from clinical trials; however, they explicitly emphasize aggregate analysis on a program level. Examples for different approaches taken by sponsors are provided. Particular considerations are advanced regarding the process of analyzing data from blinded trials, although they do not describe how to make these assessments in ongoing studies without unblinding study personnel and there are concerns about the impact of this approach on the integrity of trials. To meet the spirit of the final rule, sponsors conducting clinical trials under the authority of the FDA have responded by developing processes and tools to evaluate, assess, and act on accumulating safety information during development on an ongoing basis to ensure earlier identification of safety concerns and to take appropriate steps to mitigate risks. 23 Auspiciously, some multidisciplinary teams (advocated for in CIOMS VI) have been implementing procedures [53] [54] [55] [56] [57] [58] for review of aggregate blinded clinical trial data to support safety signal detection and risk-management activities, minimizing the need to intentionally unblind data in ongoing studies.

Recognizing that few adverse events can be judged to be drugrelated based on one or a small number of occurrences, most adverse events reported to the FDA will require an aggregate assessment to determine reasonable possibility of causal association. The motivation for the final rule, as expressed in its preamble, is a concern that unfiltered reporting of all serious adverse events (including those where there is insufficient evidence for considering them to be serious adverse reactions) could obscure relevant safety information. By establishing a process that allows all parties to focus on important safety issues, the FDA expects a substantial reduction in the number of reports containing nonmeaningful safety information. Concerned about missing important signals, EU regulators and regulators in other regions have not advocated for a reduction in SUSAR reports, preferring to receive all reports from sponsors themselves. While there are similarities, as well as some differences, regarding the need for aggregate analysis in the context of SUSAR reporting between FDA and the EU regulators, the FDA guidance documents, at this point in time, provide the most detailed considerations for aggregate assessment.

The increased emphasis on aggregate analyses of serious adverse events in order to assess causality is also reflected in the postmarketing arena. Increasingly, regulators around the world have been tapping into large electronic healthcare and claims databases that go far beyond the traditional, MAHowned serious adverse event databases. More recently, regulators have started to require MAHs to utilize these extensive new sources of information for their own postmarket surveillance due diligence.

Currently, aligned with CIOMS VIII, the EU provides the most comprehensive guidance and requirements for MAH signal detection. While Japan and the US regulators have provided less guidance, they have developed augmented sources of data for enhanced signal detection (Sentinel/ MIHARI). Globally, though significant differences remain, progress has been made in improving and harmonizing postmarketing safety surveillance across the regions.

One of the greatest challenges in the pharmaceutical industry, but at the same time, one of the greatest opportunities for growth and collaboration is that, as different region-specific regulatory initiatives go beyond ICH technical requirements, changes in emphasis develop. For postmarketing safety surveillance, aligned with CIOMS VIII, the EMA/HMA have developed numerous and diverse GVP modules that implement the 2010 EU PV legislation. Progress has been made in harmonization across regions. For clinical trial safety monitoring, aligned with CIOMS VI, the FDA has issued the IND Safety Reporting final rule (2010). The FDA has placed the responsibility squarely on sponsors to send a safety report only after they have judged there to be a reasonable possibility that the study drug caused the adverse event. For other regional regulatory agencies, including in the EU, the assessment of causality takes into account the opinion of the investigator as well as the sponsor, and all SUSARs are required to be reported in an expedited manner. Consequently, it can be challenging to apply a single global approach to report handling, and regulators in different regions could be examining different SUSAR data as trials progress, which can introduce additional challenges.

The call for aggregate reviews of accumulating safety data, including from ongoing studies, has provided an opportunity to leverage the scientific expertise and medical judgment of safety management teams with (1) a multidisciplinary approach, (2) quantitative frameworks to measure level of evidence, and (3) assessments that are product-specific and driven by medical judgment. Ongoing, interdisciplinary assessment of the accumulating data facilitates a medically informed evaluation of actual product risks. A special challenge in ongoing aggregate evaluation of safety data is the application of appropriate statistical techniques with clinical considerations, as opposed to strict statistical inference. 59 A cumulative metaanalytic review is recommended as a routine part of the safety monitoring process in order to more reliably detect and characterize increases in adverse event rates. 60 Effective safety monitoring also requires a thorough understanding of the existing safety data, the target patient population and relevant subpopulations, and risk factors for particular adverse events, as well as the product's mechanism of action, concomitant medications, and preclinical findings. The emphasis shifts from testing and confirming, to exploration with medical judgment and decision making within a quantitative framework.

There has been a steady trickle of approved medicines withdrawn from the market in response to rare but serious adverse events. Questions naturally arise as to why such important safety concerns had not been recognized before these medicines reached the market. Part of the answer can be explained by limitations imposed from the size of the safety database at the time of submission. However, it is not realistic to address this issue by simply increasing the sample size requirements for drug approval, which would have its own ethical consequences. A more useful question to ask would be, given the size of the safety database, could safety evaluation during clinical development be improved to enable earlier signal detection and management of identified risks? The response from regulators around the world has been that procedural improvements are needed for identifying patients most likely to benefit from a treatment; in particular, we have observed important trends: (1) from individual case review and snap-shot analysis of the safety data to ongoing aggregate safety evaluation, 3, 4 (2) from isolated safety analyses to benefit-risk assessments, 50,51 and (3) from separate processes to a continuum of pre-and postmarketing safety evaluation. 10, 15 Opportunity: Planning Process for Aggregate Safety Assessments throughout Clinical Development The relevance of aggregate assessments early in development has resulted in supportive activities by cross-industry expert groups. The Safety Planning, Evaluation and Reporting Team has recommended that sponsors develop a PSAP 7 as a tool to plan proactively for meta-analysis of the program safety data. Aggregate safety assessment planning and evaluations should occur throughout the life cycle of the product. Recently the DIA-ASA Interdisciplinary Safety Evaluation scientific working group has proposed an Aggregate Safety Assessment Plan (ASAP), which builds on the PSAP and addresses the FDA final rule by describing an operational framework within which accumulating safety data can be monitored and evaluated (presented at the 2018 Joint Statistical Meetings in Vancouver, British Columbia, Canada). 61 In addition, the aggregate safety assessment planning process strengthens cross-disciplinary collaboration in the planning and execution of aggregate safety assessments in order to (1) ensure a consistent approach to the collection and analysis of the safety data across the program, (2) perform ongoing, program-level signal detection and evaluation, and (3) build the "safety story" in preparation for regulatory application filing and safety-related communications.

Realizing the Value of a Planned, Systematic Approach for Ongoing Aggregate Safety Evaluations While sponsors and regulatory agencies, taking swift and decisive action to withdraw medicines from the market, have protected patients where the risks outweighed the benefits, some subsets of patients, where the benefits outweighed the risks, have been prevented from realizing the benefits of an available treatment. In some cases, new understanding has led to a targeted reintroduction of a medicine into the market, in a way that has better managed the risks and directed the drug to those indications or patient subsets where the benefit-risk profile was positive. Examples include Thalidomide, 62 Clozapine, 63 Lotronex, 64 Zelnorm, 65 Tysabri, 66 and Avandia. 67 Fedratinib, 68 which was being developed for myeloproliferative diseases, had its late-stage development program abruptly terminated in response to several cases of Wernicke's encephalopathy. Recently, an objective re-examination of the cases showed that the actual incidence of Wernicke's encephalopathy had been below the expected incidence in the target population. Aggregate safety monitoring and scientific evaluation of integrated safety data throughout development could facilitate earlier termination of programs with unacceptable risks; they could also help avoid termination of programs that demonstrate greater benefits than risks in a well-defined subpopulation (which would become the indicated patient population).

The purpose of clinical trial safety monitoring is not to eliminate risks but to identify, understand, and manage them to increase the opportunity to deliver effective medicines with favorable benefit-risk profiles to the right patients. Safety evaluation is an ongoing process starting from early development, continuing through regulatory submission and into postmarketing. A multidisciplinary team, regularly reviewing aggregate safety data throughout the development program, is vital for early signal detection, but also for generating a better understanding of the accumulating data and context needed for decreasing false alarms. In addition, such a systematic approach could lead to a more nuanced response to emerging risks, including adjustments to risk management or alterations in the target population. Interdisciplinary evaluation of accumulating safety data would empower a risk-management process that could more flexibly respond to safety issues as they emerged. By identifying distinct patient populations where the benefits outweigh the risks, perhaps with a suitable risk mitigation strategy, medicines can be better positioned in the marketplace, allowing an appropriate subset to realize important health benefits of an effective treatment.

The new emphasis on aggregate analysis for safety assessments, driven by regulators in the US and EU, leads to several challenges and corresponding opportunities for sponsor companies, as discussed above. In order to be successful, more work is needed in the following areas:

An aggregate safety assessment planning process (building on CIOMS VI and the PSAP) will enable close interdisciplinary collaboration throughout the drug development life cycle and provide for detailed planning of safety data aggregation, evaluation, and assessment. It will facilitate the following Ongoing recognition and prioritization of key safety topics of interest and characterization of the emerging safety profile for a given compound Planning and execution for ongoing aggregate monitoring (including for blinded trials), especially focused on these topics of interest Preparation for any regulatory filing activities or responses to regulatory queries To be successful, cross-disciplinary scientific engagement will be needed to integrate medical judgment and quantitative expertise in the monitoring and management of the safety profile of medicinal products. This will require team members to go beyond their functional expertise, to have openness for acquiring a better understanding of clinical and quantitative perspectives, and to strive for collaborative cross-fertilization on innovative approaches.

Numerous complex challenges exist for evaluating the relationship of study drug with occurrence of adverse events, especially when appropriately accounting for duration of exposure time, patient-level covariates, and other clinical considerations. 69 Specific safety issues (such as dose response, subgroup differences, and population exposure) have benefited from the expanded interest and participation by clinical safety professionals and statisticians working closely together. This effort continues to provide opportunities for sponsors and academia to partner with regulatory authorities for developing interdisciplinary safety evaluation procedures, in order to gain an understanding of the aggregate safety profile of drugs earlier in their development and to prepare for postmarket safety planning. More research is needed:

To develop methodologies for distinguishing ADRs from background adverse events (especially for blinded safety monitoring procedures) To improve consistency in identification and evaluation of adverse events (and other safety data) across a development program (such as ICH E9, not only for comprehensive analysis at the end of studies, but also for ongoing aggregate safety evaluations during studies) 70 To take advantage of clinical correlates in patterns of abnormality for multiple lab measurements, such as the evaluation of Drug-Induced Serious Hepatotoxicity (eDISH) plot 71 To improve the assessment of causality, develop new quantitative tools for aggregate data analysis that take into account bias and confounding, promote epidemiologic fluency among safety scientists, and provide training on qualitative frameworks, such as Bradford-Hill, 72 that can account for both quantitative and qualitative input To develop an authoritative competency framework, robust training programs, and active coordination across the industry to improve the practice, leadership, and profile of PV professionals in decision making about drug development Improving the process for detecting and characterizing signals requires supplementation of the traditional medical monitoring and medical safety review of individual cases with aggregate safety evaluations of ongoing blinded studies and cumulative meta-analyses of completed unblinded studies, from a clinical perspective, using relevant statistical analyses. As CIOMS VI states "clinical judgment is essential to decide when the threshold for adding information to the development core safety information has been reached, based on aggregate data." 6 The DIA-ASA Interdisciplinary Safety Evaluation scientific working group provides a proven working model for cross-disciplinary partnerships among sponsors, regulators, and academics.

The use of observational studies and real-world data is becoming a more integral part of PV, expanding to cover not just individual case review and spontaneous reporting but also review of accumulating safety data.

In particular, how to combine evidence from randomized controlled trials and observational/real world data for safety and benefit-risk decision-making is important to explore. Advanced analytics on multisource safety data, including artificial intelligence and machine learning methodologies, are key areas for innovative thinking. Perhaps even more importantly, design considerations to generate meaningful real-world evidence, including pragmatic randomized trials and observational studies that meet evidentiary standards for regulatory decision making, need deeper and wider exploration.

When postmarketing studies are planned to supplement information based on the preapproval safety evaluation, a robust safety profile for a drug can be managed for its life cycle, enabling better risk mitigation, effective treatment where benefits outweigh the risk, and improved patient care. Of course, the value of postmarketing studies is predicated on an actual commitment by the sponsor to adequately design and complete the study.

Interdisciplinary collaboration and communication that facilitate development of advanced data visualization techniques have become increasingly important.

Efforts are currently underway to address clinical safety needs with statistical and data science expertise along with graphical design concepts to provide dynamic, interactive, open source tools for advancing the field of signal detection and evaluation These tools are being designed in an open source model using advanced analytics supported by a workflow based on clinical experts in order to be applicable to a broad audience including industry sponsors, data-monitoring committees, and regulatory authorities Regulatory science for safety evaluation that integrates medical, statistical, and data sciences is creating substantial opportunities for interdisciplinary collaboration and innovation. This concept provides inspiration and motivation for our DIA-ASA Interdisciplinary Safety Evaluation scientific working group 73 : http://community.amstat.org/biop/workinggroups/ safety/safety-home.

This article reflects the views of the individual authors and should not be construed to represent the views or policies of their companies or the FDA.

Jonathan H. Seltzer is on the editorial board of Therapeutic Innovation & Regulatory Science.

International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use

Council for International Organizations of Medical Sciences. CIOMS homepage

Guidance for industry: safety assessment for IND safety reporting (draft

Clinical Trial Facilitation Group CTFG. Q&A document-reference safety information

Council for International Organizations of Medical Sciences. International Reporting of Adverse Drug Reactions-CIOMS Working Group Report

Council for International Organizations of Medical Sciences. Managment of Safety Information From Clinical Trials-Report of CIOMS Working Group VI

Recommendations for safety planning, data collection, evaluation and reporting during drug, biologic and vaccine development: a report of the safety planning, evaluation, and reporting team

Council for International Organizations of Medical Sciences. Practical Aspects of Signal Detection in Pharmacovigilance-Report of CIOMS Working Group VIII

International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. ICH Members and observers

International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. ICH Harmonised tripartite guideline-clinical safety data management: definitions and standards for expedited reporting E2A (current step 4 version

European Commission. Communication from the Commission -Detailed guidance on the collection, verification and presentation of adverse event/reaction reports arising from clinical trials on medicinal products for human use

Reviewer guidance-conducting a clinical safety review of a new product application and preparing a report on the review

Public workshop: Safety Assessment for Investigational New Drug Reporting

International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. ICH Harmonised guideline-revision of M4E guideline on enhancing the format and structure of benefit-risk information in ICH Efficacy-M4E (R2) (current step 4 version

International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. ICH Harmonised tripartite guideline-pharmacovigilance planning E2E (current step 4 version

Guideline for the format and content of the clinical and statistical sections of an application

Reporting cumulative proportion of subjects with an adverse event based on data from multiple studies

Reporting adverse drug reactions in product labels. Therapeutic Innovation & Regulatory Science

US Food and Drug Administration. Final rule: Investigational new drug safety reporting requirements for human drug and biological products and safety reporting requirements for bioavailability and bioequivalence studies in humans

US Food and Drug Administration

Safety reporting requirements for INDs and BA/BE studies

The FDA's final rule on expedited safety reporting: statistical considerations

The U.S. Food and Drug Administration's Mini-Sentinel program: status and direction

Active Risk Identification and Analysis (ARIA)

Questions and answers on FDA's adverse event reporting system (FAERS)

Sources of safety data and statistical strategies for design and analysis: transforming data into evidence

Clinical trial guidelines

European Medicines Agency. ICH guideline E2F on development safety update report

Directive 2001/20/EC of the European Parliament and the Council of 4

Guidance: drug safety information-FDA's communication to the public (draft

amending, as regards pharmacovigilance, Directive 2001/83/EC on the community code relating to medicinal products for human use

amending, as regards pharmacovigilance of medicinal products for human use, Regulation (EC) No 726/2004 laying down Community procedures for the authorisation and supervision of medicinal products for human and veterinary use and establishing a European Medicines Agency, and Regulation (EC) No 1394/2007 on advanced therapy medicinal products

Guideline on good pharmacovigilance practices (GVP): module IX-signal management

Guideline on the use of statistical signal detection methods in the EudraVigilance data analysis system (EMEA

Screening for adverse reactions in EudraVigilance (EMA/849944

Regulatory Information Task Force. Pharmaceutical administration and regulations in Japan

Analysis of gender and age dependent weight gain related to antipsychotic drugs by data mining of the Japanese Adverse Drug Event Report Database (JADER)

Provisions for drug registration (SFDA Order No

China National Drug Administration

Center for Drug Evaluation CFDA

Council for International Organizations of Medical Sciences. Development Safety Update Report (DSUR) Harmonizing the Format and Content for Periodic Safety Report During Clinical Trials-Report of CIOMS Working Group VII

Council for International Organizations of Medical Sciences. Evidence Synthesis and Meta-analysis-Report of CIOMS Working Group X

Council for International Organizations of Medical Sciences. International Reporting of Periodic Drug-Safety Update Summaries-Report of CIOMS Working Group II

Council for International Organizations of Medical Sciences. Benefit-Risk Balance for Marketed Drugs: Evaluating Safety Signals-Report of CIOMS Working Group IV

International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. ICH Harmonised tripartite guideline: periodic benefit-risk evaluation report (PBRER) E2C (R2, current step 4 version

International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. ICH Harmonised tripartite guideline-development safety update report E2F (current step 4 version

Continuous safety monitoring for randomized controlled clinical trials with blinded treatment information

Control charts for monitoring accumulating adverse event count frequencies from single and multiple blinded trials

Monitoring potential adverse event rate differences using data from blinded trials: the canary in the coal mine

Blinded safety signal monitoring for the FDA IND reporting final rule

A bayesian exposure-time method for clinical trial safety monitoring with blinded data. Therapeutic Innovation & Regulatory Science

Bayesian detection of potential risk using inference on blinded safety data

Willful Ignorance: The Mismeasure of Uncertainty

Meta-analyses of randomized controlled clinical trials to evaluate the safety of human drugs or biological products Guidance for industry (draft

Joint Statistical Meetings (JSM) 2018, session: drug safety monitoring in a complex world-wide regulatory environment-strategy meets methodologies

US Department of Health and Human Services, Food and Drug Administration

NDA 21107/S-025, supplement approval Lotronex (alosetron hydrochloride

NDA 21-200/S-005 Zelnorm (tegaserod maleate

BLA 125104/S-953 / BLA 125104/S-955 Tysabri (natalizumab) supplement approval, REMS modification notification

NDA 021071/S-040 / NDA 021410/S-029 / NDA 021700/S-012: Avandamet (rosiglitazone maleate and metformin hydrochloride) supplement approval

Case series of potential Wernicke's encephalopathy in patients treated with fedratinib

Statistical concepts in the planning and evaluation of drug safety from clinical trials in drug development: issues of international harmonization

The safety explorer suite: interactive safety monitoring for clinical trials. Therapeutic Innovation & Regulatory Science

Evaluation of druginduced serious hepatotoxicity (eDISH): application of this data organization approach to phase III clinical trials of rivaroxaban after total hip or knee replacement surgery

The environment and disease: association or causation?

We would like to acknowledge Janet Wittes for helping to make this regulatory review more accurate and concise.

No potential conflicts were declared.

No financial support of the research, authorship, and/or publication of this article was declared.

Greg Ball, PhD https://orcid.org/0000-0001-5626-904X