key: cord-0017219-3stcxp2x authors: Novack, Gary D. title: What is an adequate and well controlled study? date: 2021-02-17 journal: Ocul Surf DOI: 10.1016/j.jtos.2021.02.003 sha: dd8b211b62be1a52cde6ff6cea782bcd202239a8 doc_id: 17219 cord_uid: 3stcxp2x nan With the pandemic events of 2020, there was a great awareness of evaluation of novel preventative and therapeutic agents for COVID-19 infection. Upon hearing of news of a novel product, the public at large questioned the nature of the evidence. I think people might have ask questions such as "Was it just an anecdotal report in a few patients?", "Was it a large controlled trial?", and "Where was it conducted". To me, these questions all come under the heading of "Was this an adequate and well controlled study?". As scientists, in particular clinical scientists, we have several criteria for assessing the quality of a clinical study and clinical evidence. From a historical perspective, I go back to "Koch's postulates", developed by Robert Koch in the 19th century for establishing the microbiological etiology of infection and disease. These postulates are (1) The microorganism must be found in diseased but not healthy individuals; (2) The microorganism must be cultured from the diseased individual; (3) Inoculation of a healthy individual with the cultured microorganism must recapitulate the disease; and finally (4) The microorganism must be re-isolated from the inoculated, diseased individual and matched to the original microorganism [1] . These can be expanded to apply to consideration of new therapeutics. For example, in ocular surface disease, you might posit that a particular cytokine is elevated, and a treatment to reduce that cytokine might be effective. This would require steps akin to Koch's postulates. First you might show that patients with disease have elevated cytokine levels in ocular tissues, whereas healthy subjects do not. Then, you might want to show that treatment with your novel agent reduces cytokines in both normal animals and in a model of dry eye disease. You would want to show that treatment in humans (patients and/or subjects) with your novel agent reduces cytokine levels. And then finally, you would want to show that treatment in patients with this treatment decreases the signs and symptoms of disease. Today, I think that when scientists think of scientific assessment, many think of the Cochrane System. According to their website, a Cochrane Review is "...a systematic review of research in health care and health policy that is published in the Cochrane Database of Systematic Reviews. Cochrane Reviews base their findings on the results of studies that meet certain quality criteria, since the most reliable studies will provide the best evidence for making decisions about health care. Authors of Cochrane Reviews apply methods which reduce the impact of bias across different parts of the review process…" (www.cochraneli brary.com). The Cochrane method ties into "levels of evidence", which were originally described in a report by the Canadian Task Force on the Periodic Health Examination in 1979 [2] . As shown in Table 1 , this is a relatively simple 4-level scale. It ranges from expert opinion (low level) to randomized clinical trial (RCT). The levels of evidence were further described and expanded by David Sackett in an article on levels of evidence for antithrombotic agents in 1989 (Table 2 ) [3] . This is a 5-level scale, ranging from case reports (low level) to large, RCTs (plural). These scales have been expanded by many organizations, with a 10-level scale developed by the Centre for Evidence Based Medicine (CEBM) for treatment (http://www.cebm.net, Table 3 ).4. Levels of evidence is closely related to our current concepts on "Evidence Based Medicine (EBM)". Sackett in 1997 wrote EMB "… whose philosophical origins extend back to mid-19th century Paris and earlier, is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research." In a review in 2008, Masic et al. wrote"…"Evidence based medicine is the conscientious, explicit, judicious and reasonable use of modern, best evidence in making decisions about the care of individual patients. EBM integrates clinical experience and patient values with the best available research information. It is a movement which aims to increase the use of high quality clinical research in clinical decision making. EBM requires new skills of the clinician, including efficient literaturesearching, and the application of formal rules of evidence in evaluating the clinical literature. The practice of evidence-based medicine is a process of lifelong, self-directed, problem-based learning in which caring for one's own patients creates the need for clinically important information about diagnosis, prognosis, therapy and other clinical and health care issues." [5] . In the Clinical Trial Design and Regulatory subcommittee of the TFOS DEWS II report, we considered standards for well-controlled studies [6] . We cited the U.S. Congress' passage of the Kefauver-Harris amendment to the Federal Food, Drug & Cosmetic Act in 1962. That law required approval of a new drug be based upon evidence of effectiveness which is based on adequate and well-controlled clinical studies conducted by qualified experts. The definition of an adequate and well-controlled study was subsequently defined in 21 CFR 314.126. These characteristics are provided in Table 4 . The subcommittee recommended that researchers consider these attributes at the outset of clinical trial design and in selecting sites for the conduct of a clinical trial. Note that subcommittee also understood that not all studies can meet all these requirements. In particular, pilot or early stage clinical trials tend to be small, and of relatively low biostatistical power. These characteristic however are recommended for pivotal, Phase 3 studies. The conduct of clinical trials is provided in detail in "Good Clinical Practice" (GCP). Recommendations for the conduct of trials according to GCP are provided in detail in the International Conference on Harmonisation (ICH) "Guideline For Good Clinical Practice (E6, https://data base.ich.org/sites/default/files/E6_R2_Addendum.pdf). Extramural studies sponsored by the National Institutes of Health do not necessarily The European Union (and European Medicines Agency) do not seem to use the same terms as the U.S. FDA. However, they do refer to the "reliability and robustness" of the clinical trial and its data, based on adequate monitoring of the trial; traceability, storage, return and destruction of investigational medicinal products; adequate recording, storage and handling of the clinical trial data; rules on manufacturing, importation and distribution of investigational and auxiliary medical products to clinical trial sites; appropriate labeling of investigational medical products; the quality of the investigational medicinal products; data protection and informed consent; adherence to GCPs; statistical approaches, design of the clinical trial and methodology, including sample size and randomization, comparator and endpoints (Regulation 536/2014, https://ec.europa.eu/health/human-use/clinical-trials/ regulation_en).China also follows ICH E6. In summary, there are many published, accepted standards for the definition of an adequate and well controlled study. These definitions then lead us to the practice of evidence based medicine. This "science of medicine" is often in contradistinction to the "art of medicine", which is often driven by anecdotal clinical experience. This is especially true in ocular surface disease, in which there is a wide variability of patient signs and symptoms, and thus the need to individualize therapy. Nonetheless, it is adequate and well controlled studies that are used by regulators to approve novel therapies for marketing, so that they then become available for patients. Ophthalmic products related to the ocular surface preclinical toxicology study of intravitreal EYP-1901 (a proprietary delivery system of vorolanib), intended to support evaluation in patients wet age-related macular degeneration (wet AMD, December 2020). The firm also started enrollment in its Phase 1 study of intravitreal EYP-1901 (vorolanib, a tyrosine kinase inhibitor, in its bioerodible Durasert® delivery system) in patients with wet AMD (January 2021). Case series (and poor quality cohort and case-control study 5 Expert opinion without explicit critical appraisal or based on physiology bench research or "first principles" RCT = Randomized clinical trial. From [4] . Definition of an adequate and well controlled study. (1) Clear statement of the objectives of the investigation. (2) Uses a design that permits a valid comparison with a control to provide a quantitative assessment of drug effect. (i) Placebo, (ii) Dose-comparison, (iii) No treatment (iv) Active (v) Historical control. (3) Method of selection of subjects provides adequate assurance that they have the disease or condition being studied. (4) Method of assigning patients to treatment and control groups minimizes bias. (5) Adequate measures are taken to minimize bias on the part of the subjects, observers, and analysts of the data. (6) Methods of assessment of subjects' responses are well defined and reliable. (7) There is an analysis of the results of the study adequate to assess the effects of the drug. (d) The test drug is standardized as to identity, strength, quality, purity, and dosage form to give significance to the results of the investigation. • Gemini Therapeutics received Fast Track designation for its GEM103 (a recombinant native complement modulator, full-length recombinant complement factor H protein) (January 2021). The firm also started a Phase 2a trial of GEM103 in patients with wet AMD (February 2021). • Genentech/Roche announced results from four Phase 3 programs in which its intravitreal bispecific antibody, faricimab, was evaluated: YOSEMITE and RHINE in patients with diabetic macular edema (December 2020), and TENAYA and LUCERNE in patients with wet AMD (January 2021). for its Dextenza® (dexamethasone ophthalmic insert) for the treatment of ocular itching associated with allergic conjunctivitis (December 2020). • Ocuphire completed enrollment in its MIRA-2 Phase 3 study of its Nyxol® (phentolamine) to reverse pharmacologically-induced mydriasis (January 2021). The firm also started its LYNX-1 study evaluating the safety and efficacy of Nyxol® in night vision disturbances (January 2021). • ONL Therapeutics will be developing ONL 1204 (a Fas inhibitor) for treatment of retinal detachment, glaucoma and dry AMD (December 2020). • PolyActiva completed a Phase I clinical study for its Latanoprost FA SR Ocular Implant for the treatment of elevated introacular pressure (November 2020). • REGENXBIO started enrollment in its ALTITUDE study, a Phase II trial to evaluate the suprachoroidal delivery of RGX-314 gene therapy using the suprachoroidal space microinjector for the treatment of diabetic retinopathy (DR, December 2020). • SIFI completed enrollment of its Phase 3 trial for evaluation of polihexanide in patients with Acanthamoeba keratitis (November 2020). • Atsena Therapeutics will be developing gene therapy for GUCY2Dassociated Leber congenital amaurosis (LCA1) disease (December 2020). • Foundation Fighting Blindness listed 42 ongoing trials of therapeutics in inherited retinal disease and dry AMD (January 2021). • Janssen purchased Hemera Biosciences which is developing a CD59 gene therapy for the treatment of geographic atrophy (December 2020). • Lineage Cell Therapeutics completed enrollment in a Phase 1/2a study of OpRegen® (a subretinal investigational retinal pigment epithelium cell therapy) in patients with geographic atrophy (November 2020). Other news about pharmaceutical and medical device firms • AstraZeneca received a positive recommendation from the European Medicine's Agency for its COVID-19 vaccine (January 2021). • Several firms recalled oral metformin products due to the detection of N-Nitrosodimethylamine (October 2020 to January 2021). • The United Kingdom National Institute for Health and Care Excellence (NICE) issued a final appraisal determination recommending Novartis' brolucizumab (Beovu®) treatment for wet AMD (December 2020). • The U.S. National Institutes of Health encouraged rapid, collaborative reporting of clinical trials results with COVID-19 therapies on clinicaltrials.gov (November 2020). • In the relatively few serious allergic reactions to COVID-19 vaccine injections, there may be a connection to polythelyne glycol (PEG), an excipient also used in ophthalmic products (December 2020). • The U.S. Department of Health and Human Services (of which FDA is a part) highilighted their significant concerns about companies offering or paying remuneration to health care professionals in connection with speaker programs (November 2020). • As dermatologists expand the off-label use of topical dermal timolol (using the ophthalmic product), there is a growing realization of cardiopulmonary adverse events (well known to the ophthalmic community, January 2021) [7] . o Terminated a program regarding the removal of unapproved drugs after an approval of the same drug, as well is considering how to deal with "grandfathered drugs" (marketed prior to 1938) [8] . o Is reorganizing the office of generic drugs to strengthen its operations and allow the office to meet the evolving needs of generic drug review. o Providing direction on increasing the sterility and other aspects of pharmacy compounded pharmaceuticals. o Announced focus areas for regulatory science. o Published guidances for Developing drugs for treatment of dry eye disease. o Published other guidances include best practices in developing proprietary names for human prescription drug products, biosimilarity and Interchangeability, enhancing the diversity of clinical trial populations, investigational new drug applications for Individualized antisense oligonucleotide drug products [9] , and protecting participants in bioequivalence studies for generic drugs during the COVID-19 health emergency. What does it take to satisfy Koch's postulates two centuries later? Microbial genomics and Propionibacteria acnes The periodic health examination. Canadian Task Force on the periodic health examination Rules of evidence and clinical recommendations on the use of antithrombotic agents The levels of evidence and their role in evidencebased medicine Evidence based medicine -new approaches and challenges TFOS DEWS II clinical trial design report Adverse effects of topical timolol: safety concerns and implications for dermatologic use Of the Food and drug administration's unapproved drugs initiative; request for information regarding drugs potentially generally recognized as safe and effective Drug regulation in the era of individualized therapies Gary D. Novack PhD consults with numerous pharmaceutical firms.