key: cord-0852188-71e4qcxw authors: Yang, Youyang; Mehta, Nilesh M. title: “Closing the chasm” – guidelines bridge the gap from evidence to implementation date: 2021-09-22 journal: Pediatr Investig DOI: 10.1002/ped4.12296 sha: 1a74bfdd517aefde6be8f156ef642b8725d2fcc4 doc_id: 852188 cord_uid: 71e4qcxw Since the beginning of the Coronavirus Disease 2019 (COVID-19) pandemic, over 9000 articles related to COVID-19 have been released in print by the end of December 2020. For guidelines to be impactful, they must be perceived as a credible source of recommendations for the delivery of high-quality care, based on rigorous and transparent appraisal of the evidence available. [...]the process of guideline development should be standardized and transparent. [...]the NSF-LTC framework is advantageous for long-term conditions that may not be amenable to rigorous RCTs, thereby allowing for a more varied research base, including longitudinal, case-report and qualitative studies. [...]the timely implementation of the recommendations in guidelines remains challenging. Since the beginning of the Coronavirus Disease 2019 (COVID-19) pandemic, over 9000 articles related to COVID-19 have been released in print by the end of December 2020. 1 A majority of these articles were commentaries, several hundred were observational studies, 55 were systematic reviews, and 4 were randomized controlled trials. The American College of Rheumatology has published updated clinical guidance for the management of multisystem inflammatory syndrome in children associated with SARS-CoV-2, the most recent version in April 2021. 2, 3 Additionally, practice guidelines for pediatric specialties in the setting of COVID-19 have also been published, including pulmonology, anesthesiology, infectious disease, orthopedics, and anticoagulation. [4] [5] [6] [7] [8] The rapid development and dissemination of these guidelines during the pandemic has been unprecedented. Many of these guidelines are based primarily on consensus statements rather than randomized control trials, which have been the gold standard of evidence-based medicine. In this commentary, we discuss the role of practice guidelines from credible sources that are based on incrementally emerging evidence to guide bedside practice, until they can be updated with more robust data from meta-analyses of randomized trials. The Institute for Healthcare Improvement (IHI) framework for safe, reliable and effective care includes nine components that are split into two categories: "Closing the chasm" -guidelines bridge the gap from evidence to implementation DOI: 10.1002/ped4.12296 This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made. learning system and culture. One of the nine components of the IHI framework, reliability in the learning system, refers to applying best evidence and minimizing variation in practice that can be achieved by developing and implementing evidence-based guidelines. 9 It is imperative that these guidelines are supported by best available and relevant evidence. However, high-quality evidence for many questions in pediatric critical care is not available. In the absence of strong recommendations with the highest quality of evidence, is an evolving consensusbased guideline that is updated over time better than no guideline? In a recent review of guidelines from 2011 to 2017 published by the Society of Critical Care Medicine, there was discordance between strength of recommendations and its associated quality of evidence in a considerable proportion of the 681 recommendations. 10 About 32.1% of the strong recommendations were backed by low quality evidence or expert consensus only. A uniform process of guideline development, from searching and grading of evidence to the development of clear and evidence-based practical recommendations, is critical to the credibility of the recommendations and their utility in healthcare quality improvement. For guidelines to be impactful, they must be perceived as a credible source of recommendations for the delivery of high-quality care, based on rigorous and transparent appraisal of the evidence available. Therefore, the process of guideline development should be standardized and transparent. The methodology must ensure concordance between the level of evidence and the strength of the recommendation. Traditionally, guideline development starts with commissioning an interdisciplinary team of experts in the field of interest, with support from one or more professional organizations that approve the development process and will eventually review and endorse the recommendations. The team develops clinically relevant key questions to be addressed by the guidelines. The overarching topic for the guideline is broad but consists of narrowly defined questions that would have practical application in specified populations and healthcare settings. The questions typically center around the impact of interventions on outcomes of interest in a specified population. A comprehensive and standardized literature search is then performed. This typically begins with a description of the populations of interest, the database(s) used for the search, the exact terms used in the search, and the study selection criteria. For example, in a guideline that examines prognosis, studies included in the appraisal should have mortality as a researched outcome. The guideline team must a priori identify the framework used to grade the recommendations as strong or weak, based on the balance between and the degree of confidence in estimates of benefits, risks, and burden. Costs or resource utilization is also an important consideration when grading the recommendations. After the studies are identified, the responsibilities of the review process are distributed amongst team members, often with multiple members reviewing the same articles under pre-established rules for determining interrater reliability. Each study is reviewed and scored objectively by one or more members of the team in accordance with the pre-determined framework of critical appraisal. A variety of software and online tools have been developed to facilitate the process of uploading, reviewing, scoring, sharing of studies during this process of evidence synthesis, and are highlighted in Table 1 . 11 Critical appraisal is typically focused on two features: 1) strength of the recommendation and 2) quality of evidence behind the recommendation. The evidence is then critically appraised, and the quality/strength/level of evidence is determined based on factors such as the study design, rigor of the study, types of outcomes reported and the precision of estimates. Meta-analyses of randomized control trials have traditionally been regarded as gold standard for evidence-based. However, the lack of well-designed large RCTs to address most questions in pediatrics and neonatology, result in a significant challenge for guidelines development. The resultant recommendations are therefore assigned a low grade. Some questions may need to be answered by well-designed observational studies with statistical rigor. The quality of an individual study may be downgraded for limitations in study design or implementation, wide confidence intervals, variability in results, and publication or other biases. On the other hand, quality may be upgraded for some factors such as a large magnitude of effect with a dose-response relationship. Quality of individual studies are collated and synthesized to help develop a series of recommendations. A variety of grading systems are available with their individual strengths and weaknesses. 12 The potential strengths and drawbacks of different grading systems were explored in a review of the Scottish Intercollegiate Guidelines Network (SIGN), the Grading of Recommendations, Assessment, Development and Evaluation (GRADE), the Graphic Appraisal Tool for Epidemiology (GATE) and the National Service Framework for Long Term Conditions (NSF-LTC) grading systems. 12 SIGN is a checklistbased tool that is commonly used, simple, and focuses on reducing study and appraiser bias. The GRADE approach was developed through rigorous evaluation of other grading systems. In particular, contrary to other grading systems, it acknowledges potentially weak RCTs and potentially strong observational studies. Additionally, GRADE takes into consideration the net benefits and costs of each recommendation. 13, 14 The GATE framework is simple to use, though it is primarily designed to critically appraise a study but not assist in assigning grades or recommendations. Finally, the NSF-LTC framework is advantageous for long-term conditions that may not be amenable to rigorous RCTs, thereby allowing for a more varied research base, including longitudinal, case-report and qualitative studies. The review suggests unique utility to each grading framework depending on the type of review question posed (e.g., therapy, diagnosis, screening, prognosis, causation, etc.). 12 Once the guidelines are formed using the pre-determined framework, dissemination typically occurs through publication in a journal typically of high impact in the professional society in which the guideline applies. In the current digital age, dissemination also occurs through social media platforms, digital journal publications, and online educational portals. The devastating and rapid onslaught of the COVID-19 pandemic necessitated equally rapid generation of guidelines to avoid wide variation in practice due to the knowledge gap. Practical guidelines were developed based on observational studies, small trials and expert consensus in the absence of high-quality evidence. These guidelines were modified and incrementally fine-tuned as new evidence from larger trials became available. 15 Therefore, if guidelines are released primarily with recommendations based on suboptimal evidence or consensus; they should clearly indicate the evidence level and strength of recommendations. Furthermore, it should then be mandatory to standardize a timely process for re-evaluation of such guidelines in order to incorporate emerging new evidence. Lastly, the timely implementation of the recommendations in guidelines remains challenging. The insightful document of the Institute of Medicine, "Crossing the Quality Chasm", described the six aims for healthcare improvement: Safety, Effectiveness, Patient-Centeredness, Efficiency, Equity and Timeliness. 16 One of the greatest challenges in achieving the aims of timeliness and equity, is the ability to implement practice guidelines expediently and widely across different clinical systems. Even when guideline recommendations are developed based on strong evidence, it can take on average 17 years for them to be incorporated into routine practice, and only about half of such practices reach widespread clinical usage. 17 Given the current barriers in implementing guidelines in a timely fashion, future efforts should focus on methodology to expedite the implementation process, as well as standardized periods of mandatory updates to the guideline to keep up with the ever-evolving body of evidence. The COVID-19 global pandemic has proven that the healthcare and research community can work at a faster pace, conducting large-scale pragmatic trials followed by rapid cycle implementation of best practice recommendations. 18, 19 Furthermore, all of the previous steps can occur in parallel with research studies that then generate recommendations based on high quality evidence. The expedited development, rapid funding and collaborative conduct of large clinical therapeutic trials for COVID-19 allowed evidence-based therapies to be instituted within a short span of time. This model should guide future efforts at scientific discovery, guideline development and their implementation. In summary, guidelines play an important role in bridging the gap between incrementally accumulating evidence and the implementation of best practices at the bedside. There continues to be a dearth of highquality evidence for many of the clinical questions in neonatology and pediatric practice. Guidelines that adhere to sound and uniform process of evidence appraisal and grading of recommendations by expert consensus may have an important role. They could provide best practice recommendations based on the existing level of evidence and can be modified as new evidence emerges. There may be benefits to eliminating the variations in aspects of healthcare delivery with clear evidence for best practice. The transparent and uniform reporting and grading of the recommendations and the commitment to incorporate new evidence in a timely fashion will provide credibility to these guidelines. Such guidelines facilitate wide dissemination of the synthesized knowledge that is distilled into practical recommendations, with wide buy-in, especially when they are endorsed by reputed interdisciplinary organizations. They may also illustrate the gaps in evidence and identify priorities for future research. Overall, such a process of guideline development and dissemination must be followed by the implementation of the recommendations at the bedside. Implementation science is the crucial step in a learning healthcare system (LHS) where evidence generation is continually embedded in bedside care. This will require broad commitment and investment, as with the National Institute of Health's funding opportunities in Dissemination & Implementation Research. 20 Only by completing the LHS cycle will evidence-based guidelines achieve the promised benefit of delivering high-quality care to critically ill patients everywhere. COVID-19 and literature evidence: should we publish anything and everything? American College of Rheumatology Clinical Guidance for Multisystem Inflammatory Syndrome in Children Associated With SARS-CoV-2 and Hyperinflammation in Pediatric COVID-19: Version 1 American College of Rheumatology Clinical Guidance for Multisystem Inflammatory Syndrome in Children Associated With SARS-CoV-2 and Hyperinflammation in Pediatric COVID-19: Version 2 Pediatric Airway Management in COVID-19 Patients: Consensus Guidelines From the Society for Pediatric Anesthesia's Pediatric Difficult Intubation Collaborative and the Canadian Pediatric Anesthesia Society Practical guide for pediatric pulmonologists on imaging management of pediatric patients with COVID-19 Consensus-based clinical recommendations and research priorities for anticoagulant thromboprophylaxis in children hospitalized for COVID-19-related illness Recommendations for the Care of Pediatric Orthopaedic Patients During the COVID-19 Pandemic Caring for children and adolescents with type 1 diabetes mellitus: Italian Society for Pediatric Endocrinology and Diabetology (ISPED) statements during COVID-19 pandemia A Framework for Safe, Reliable, and Effective Care. White Paper. Cambridge, MA: Institute for Healthcare Improvement and Safe & Reliable Healthcare Above the GRADE: Evaluation of Guidelines in Critical Care Medicine Systematic Reviews: Tools & Resources A review of grading systems for evidence-based guidelines produced by medical specialties Understanding GRADE: an introduction Grading quality of evidence and strength of recommendations Chloroquine or hydroxychloroquine for prevention and treatment of COVID-19 Crossing the Quality Chasm: A New Health System for the 21st Century An introduction to implementation science for the nonspecialist Guidance and patient instructions for proning and repositioning of awake, nonintubated COVID-19 patients Front-Line innovation: Rapid implementation of a nurse-driven protocol for care of outpatients with COVID-19 National Institute of Health: Office of Disease Prevention. Dissemination & Implementation None.