key: cord-0734395-vbo73ybm authors: Akl, Elie A.; Morgan, Rebecca L.; Rooney, Andrew A.; Beverly, Brandiese; Katikireddi, Srinivasa Vittal; Agarwal, Arnav; Alper, Brian S.; Alva-Diaz, Carlos; Amato, Laura; Ansari, Mohammed T.; Brozek, Jan; Chu, Derek K.; Dahm, Philipp; Darzi, Andrea J.; Falavigna, Maicon; Gartlehner, Gerald; Pardo-Hernandez, Hector; King, Valerie; Klugarová, Jitka; Langendam, M. W. Miranda; Lockwood, Craig; Mammen, Manoj; Mathioudakis, Alexander G.; McCaul, Michael; Meerpohl, Joerg J.; Minozzi, Silvia; Mustafa, Reem A.; Nonino, Francesco; Piggott, Thomas; Qaseem, Amir; Riva, John; Rodin, Rachel; Sekercioglu, Nigar; Skoetz, Nicole; Traversy, Gregory; Thayer, Kris; Schünemann, Holger title: Developing trustworthy recommendations as part of an urgent response (1-2 weeks): a GRADE concept paper GRADE Rapid Guidelines project group date: 2020-09-30 journal: J Clin Epidemiol DOI: 10.1016/j.jclinepi.2020.09.037 sha: 76974befc008d4e57e56d6cff869023adc7fba54 doc_id: 734395 cord_uid: vbo73ybm Objective To propose an approach for developing trustworthy recommendations as part of urgent responses (1-2 weeks) in the clinical, public health, and health systems fields. Study design and setting We conducted a review of the literature, outlined a draft approach, refined the concept through iterative discussions, a workshop by the GRADE Rapid Guidelines project group, and obtained feedback from the larger GRADE working group. Results A request for developing recommendations within two weeks is the usual trigger for an urgent response. While the approach builds on the general principles of trustworthy guideline development, we highlight the following steps: (1) assess the level of urgency; (2) assess feasibility; (3) set up the organizational logistics; (4) specify the question(s); (5) collect the information needed; (6) assess the adequacy of identified information; (7) develop the recommendations using one of four potential approaches: adopt existing recommendations; adapt existing recommendations; develop new recommendations using existing adequate systematic review; or develop new recommendations using expert panel input; and (8) consider an updating plan. Conclusion An urgent response for developing recommendations requires building a cohesive, skilled and highly motivated multidisciplinary team with the necessary clinical, scientific, and methodological expertise, adapting to shifting needs, complying with the principles of transparency and properly managing conflicts of interest. time-consuming tasks are the identification, synthesis, assessment and presentation of the evidence in ways that allow the guideline groups to formulate recommendations. Over the last two decades, the GRADE working group has developed a systematic and transparent approach to creating evidence-informed practice guidelines that facilitate this. The method has been applied across the clinical, public health and health systems fields (3) (4) (5) . Some situations raise the need for urgent recommendations to support the interventions of clinicians, public health practitioners, and policymakers, for example to address the COVID-19 pandemic. (6, 7) Under such circumstances, developing trustworthy recommendations in a sufficiently short timeframe is essential but can be challenging in rapidly changing contexts. Groups producing guidelines specifically need to balance the need for developing a timely response with the need to ensure the trustworthiness of their advice. The World Health Organization's (WHO) defined two types of guidelines developed in response to an emergency or urgent need: emergency (rapid response) guidelines (produced within hours to days), and rapid advice guidelines.(8) Thayer and Schünemann defined four levels of urgency for developing recommendations: ultra-short emergency response (1-2 hours), urgent response (1-2 weeks), rapid response (1-3 months) and routine response (more than 3 months) J o u r n a l P r e -p r o o f (Box 1) (6) . While more detailed advice exists for routine (1) and rapid responses, so far there is no formal guidance on how to apply GRADE in situations requiring urgent responses (6, 9) . Box 1: Levels of urgency for developing recommendations The objective of this paper is to propose an approach for developing trustworthy recommendations as part of urgent responses (1-2 weeks) in the clinical, public health, and health systems fields. The target audience for the proposed approach are guideline developers. We consider the timeframe from the perspective of the guideline developer, i.e., the starting point for the J o u r n a l P r e -p r o o f We debated, revised and refined the concepts to be included in the approach based on iterative discussions during conference calls and a workshop by the project group (Rome, Italy, April 27, 2017) . Members of the project group also drew on their experiences with developing recommendations in response to rapid or urgent needs (17) (18) (19) . Finally, the GRADE working group approved this paper using its expedited publication approval process, developed in light of the COVID-19 pandemic, and the need for development of trustworthy recommendations for its management internationally as part of urgent responses. The project group agreed to build the urgent response approach on the commonly accepted principles of standard guideline development (e.g., conflict of interest management). (1, 2) The group also agreed that the urgent response approach can also build on the GIN-McMaster guideline development checklist extension for rapid recommendations (see table here: https://bit.ly/rapidGDC). (11) Ideally, the process should include the conduct of a rapid review, whenever feasible. However, and given the major challenges with feasibility, in the subsequent sections we do not consider a rapid review as an option. The proposed approach assumes adherence to the criteria for applying or using GRADE. (20) These criteria include giving explicit consideration to the GRADE factors for determining the direction and strength of a recommendation and, ideally, using GRADE evidence to decision (EtD) frameworks and tables (21) . Under standard guideline development conditions, the EtD The following paragraphs discuss the steps of the urgent response approach and assume that existing and adequate guidelines and systematic reviews, but not new rapid reviews, are being considered. However, if a rapid review is performed it may be added at step 5 in this process. We provide illustrative examples of urgent situations related to environmental exposures and to the COVID-19 pandemic. The purpose of this first step is to confirm that the response in developing the recommendation should be an 'urgent' one, as opposed to 'emergent' or 'rapid' (see box 1). While there is no tool specifically developed for that purpose, one can use tools such as the WHO rapid risk assessment of acute public health events. (24) Risk assessment includes the assessment of the likelihood of the occurrence of a hazard causing an adverse event, as well as the likely magnitude of the consequences of the event over a specified period. It is important for the guideline developers to assess the feasibility of meeting the 'urgent' timeframe (i.e., 1-2 weeks). Aspects to be considered include: 1) the number and complexity of questions and comparisons; 2) the expected amount of work needed for collecting the literature; 3) the number of team members available to contribute to the different tasks; 4) sources of funding that can be promptly tapped to support the urgent response; 5) the availability of adequate coordination capacity; 6) the panelists who can be summoned to contribute to collecting the needed information, panel discussion, and peer review in a very intensive manner; and 7) the political and institutional support to expedite the administrative process -collaboratively with the relevant endorsing bodies -(particularly the approval of the final product and if needed, of the dissemination and implementation strategies), if applicable. As the guideline developers start working with the requestors on specifying the questions (see To make the process feasible, the guideline developers should work with the requestor to ensure the guideline's scope is reasonably narrow. In addition, the guideline developers should ensure the questions are specific and well defined, e.g., using the PICO (Population, As mentioned earlier, the proposed approach considers using existing guidelines and systematic reviews and does not necessarily require the conduct of new rapid reviews. NICE has adopted In addition to information about the health effects, the guideline developers may need to identify contextual information. In the example of mass evacuation in response to a nuclear incident, there is a need to collect information about the cost and feasibility of such a major undertaking, as well as the acceptability by relevant stakeholders (e.g., governmental agencies, citizens). This information may be available from administrative databases (e.g., for cost), and through consulting with community leaders (e.g., to assess the acceptability by stakeholders). In terms of information sources, the published literature may not be the only or optimal source in urgent and rapidly developing situations, like the COVID-19 pandemic. Information sources may include preprint servers, open research databases, and research data repositories. However, such sources may not include conventional processes of peer review and may require J o u r n a l P r e -p r o o f additional critical appraisal or caution in the interpretation of the accuracy or stability of the data. Specific to environmental exposure scenarios, it may be important to evaluate or model the evidence for health effects when little is known about the exposure (e.g., a chemical with uncertain effects). Once identified, the published literature (guidelines, systematic reviews and if applicable, primary studies of any design) needs to be assessed for 'adequacy' based on the three following criteria: relevance, credibility, and currency.(32) • Relevance or directness, according to GRADE terminology, (33) refers to the extent to which the population, exposure/intervention, and outcomes identified in the literature (whether for the original recommendation or for the systematic reviews) reflect the ones in the question being addressed by the recommendation. If any items are different enough that we would expect different relative effects, then the evidence is considered indirect. • Credibility reflects the quality of conduct of the guideline or of the systematic review being considered for use in the process of the urgent response. There are a number of tools that could be used for that purpose: AGREE II to assess the quality of conduct of guidelines;(34) AMSTAR-2 to assess the quality of systematic reviews, (35) and ROBIS to assess the risk of bias of systematic reviews. The adequacy of identified literature is not an "all or none" assessment. For example, the guideline developer might identify three potentially relevant systematic reviews. Each one of the three may be deficient in relation to one of the three above criteria, but to different extents. We suggest considering the criteria in the following order: relevance, credibility, and currency. This means that the guideline developers would select a relevant but outdated systematic review over an updated, but less relevant review. The idea is that it is more efficient to 'fix' the former review with a quick update than to use the latter less relevant review. When no adequate recommendations are available for adopting or adapting, and no adequate systematic reviews are available to inform a de novo recommendation, using expert panel input as the sole source of evidence is a feasible alternative for an urgent response. It is important to clarify that panel input is still important when developing the recommendation based on existing recommendations or systematic reviews. the chair can make suggestions and have panelists agree or disagree with them; however, caution is needed to ensure a broad range of perspectives is considered. Urgent situations are typically associated not only with scarcity of data but also with rapidly developing evidence base and contextual information. This raises the consideration of establishing an updating plan, ideally through a living process. (8, 45) This is particularly relevant when emerging data can potentially lead to a change in the recommendation. The updating plan would need to define the frequency of reassessment of the recommendation (e.g., weekly, monthly), and to be adjusted to the speed of development of the urgent situation and of the emergence of the evidence. In the case of an urgency related to an environmental exposure, there might be a need to continuously monitor the level of exposure within the population of interest for the purpose of triggering or updating the recommendations. (45) In the example of the nuclear incident, regular collection of environmental radioactive iodine levels from the field would be needed to reverse the recommendation for mass evacuation. Developing trustworthy guidelines in a relatively short timeframe are of utmost importance but can be extraordinarily challenging. In this paper we propose an approach for developing trustworthy recommendations as part of an urgent response (1-2 weeks). The approach offers the alternatives of using existing guidelines to adopt or adapt recommendations; using existent systematic reviews to develop new recommendations; and, when the previous options are not possible, relying on expert panel input to develop new recommendations. The alternatives of adopting, adapting, and using existent reviews to developing new recommendations are similar to the ones proposed by the GRADE Adolopment methodology. (46) While that methodology was designed to address guideline development primarily to save and share resources, the approach proposed in this paper is intended to address limited available time (and also potentially limited resources, recognizing that the absence of guideline recommendations may pose more of a threat than rapidly developed ones The use of the PICO versus the PECO framework is dependent on the knowledge gap being addressed. For instance, when little is known about the outcomes arising from the exposure to a potential hazard, a PECO question is needed. Conversely, if the exposure is known to be a hazard, and information is needed about the effectiveness and safety of a mitigating intervention, a PICO question is needed (27) . • Population: includes those whose health outcomes might be affected (positively or negatively) by either the exposure in a PECO question (e.g., exposed to the released radioactivity during a nuclear incident); or the management options in a PICO question (e.g., mass evacuations or providing iodine following a nuclear incident). For the COVID-19 pandemic five populations of interest have been described: people known to be infected, people presumed to be infected, people exposed to those infected, people not known to be exposed or infected, and people who have recovered from infection. (49) • Exposure/Intervention and comparator(s): these terms refer to either classifying the potential hazard (exposure) or, to classifying the management options (interventions) under consideration. In some cases, there is a need to define the exposure level above which an action would be triggered and/or below which an action could be discontinued. It is important to note that interventions may consist of a comprehensive, coordinated response consisting of many components. Consequently, it is important to J o u r n a l P r e -p r o o f clarify the specific components, timing and duration of their delivery, and the individuals involved (27) . In the nuclear incident example, when considering the intervention of providing iodine versus not providing iodine, the following needed to be specified: the dosage and number of doses of prophylactic iodine; and the timeframe within which iodine should be ingested. • Outcomes: These should ideally include all critical and important health outcomes that the alternative management options are expected to influence impact. In the above example, these could include the health and psychosocial effects of radioactive iodine exposure, and the benefits and adverse effects of prophylactic iodine therapy. Healthrelated outcomes should be prioritized based on patient importance. In some cases, policy makers might be interested in the impact on non-health outcomes, e.g., socioeconomic impact of mass evacuation and road accidents caused by mass Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise Guidelines International Network: toward international standards for clinical practice guidelines GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines The GRADE Evidence to Decision (EtD) framework for health system and public health decisions. Health research policy and systems The GRADE evidence-to-decision framework: a report of its testing and application in 15 international guideline panels Using GRADE to respond to health questions with different levels of urgency WHO Rapid Advice Guidelines for pharmacological management of sporadic human infection with avian influenza A (H5N1) virus Guidance for updating clinical practice guidelines: a systematic review of methodological handbooks Developing WHO rapid advice guidelines in the setting of a public health emergency World Health Organization. Guidelines for WHO guidelines. Geneva: World Health Organization Development of rapid guidelines: 2. A qualitative study with WHO guideline developers Development of rapid guidelines: 1. Systematic survey of current practices and methods Development of rapid guidelines: 3. GIN-McMaster Guideline Development Checklist extension for rapid recommendations Clinical management of severe acute respiratory infection when novel coronavirus ( 2019-nCoV) infection is suspected: interim guidance GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo J o u r n a l P r e -p r o o f development of trustworthy recommendations: GRADE-ADOLOPMENT Infection Prevention and Control guidance for Long-Term Care Facilities in the context of COVID-19: interim guidance Infectious Diseases Society of America Guidelines on the Treatment and Management of Patients with COVID-19 AGA Institute Rapid Recommendations for Gastrointestinal Procedures During the COVID-19 Pandemic Should Clinicians Use Chloroquine or Hydroxychloroquine Alone or in Combination With Azithromycin for the Prophylaxis or Treatment of COVID-19? Cardiac Involvement in a Patient with Coronavirus Disease 2019 (COVID-19) GRADE Evidence to Decision (EtD) Frameworks: A Systematic and Transparent Approach to Making Well Informed Healthcare Choices. 1: Introduction Cochrane Rapid Reviews Interim Guidance from the Cochrane Rapid Reviews Methods Group 2020. 2020. 23. Clinical Characteristics of Children with Coronavirus Disease Rapid Risk Assessment of Acute Public Health Events. World Health Organization Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation Identifying the PECO: A framework for formulating good questions to explore the association of environmental and other exposures with health outcomes Analysis of Epidemiological and Clinical features in older patients with Corona Virus Disease 2019 (COVID-19) out of Wuhan Surviving Sepsis Campaign: guidelines on the management of critically ill adults with Coronavirus Disease 2019 (COVID-19) Statement of EFSA on risks for public health due to the presences of melamine in infant milk and other milk products in China Adaptation of the 2015 American College of Rheumatology treatment guideline for rheumatoid arthritis for the Eastern Mediterranean Region: an exemplar of the GRADE Adolopment GRADE guidelines: 8. Rating the quality of evidence--indirectness AGREE II: advancing guideline development, reporting and evaluation in health care AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both ROBIS: A new tool to assess risk of bias in systematic reviews was developed National Toxicology Program. National Toxicology Program research project: West Virginia chemical spill Expert Evidence in Guidelines G. Distinguishing opinion from evidence in guidelines Ventilation Techniques and Risk for Transmission of Coronavirus Disease Ann Intern Med. 2020. 42. group Aw. Use of Ebola Vaccine in the United States: Recommendations of the Advisory Committee on Immunization Practices -United States, 2020. Morbidity and Mortality Weekly. under review Clinical management of severe acute respiratory infection when Middle East respiratory syndrome coronavirus ( MERS-CoV) infection is suspected: interim guidance. World Health Organization SARS: systematic review of treatment effects Living systematic reviews: 4. Living guideline recommendations GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo development of trustworthy recommendations: GRADE-ADOLOPMENT Evidence vs Consensus in Clinical Practice Guidelines Public Health Protection in Radiation Emergencies. PHE publication. 2019. 49. Fineberg HV. Ten Weeks to Crush the Curve