key: cord-0908927-2spugw99 authors: Garritty, Chantelle M.; Norris, Susan L.; Moher, David title: Developing WHO rapid advice guidelines in the setting of a public health emergency date: 2016-08-31 journal: J Clin Epidemiol DOI: 10.1016/j.jclinepi.2016.08.010 sha: 998548cf6bc83b4dfc9d061a2a4425cc9506e82e doc_id: 908927 cord_uid: 2spugw99 OBJECTIVES: We describe newly established guidance for guideline developers at the World Health Organization (WHO) on the process and procedures for developing a rapid advice guideline in the context of a public health emergency (e.g., the 2014 Ebola epidemic). STUDY DESIGN AND SETTING: We based our approach on established rapid review methods, which were incorporated into existing WHO guideline development processes. Guidance was further informed by in-depth discussions of issues related to rapid guideline development with WHO staff (n = 6), who oversee the Organization's response to emergencies. RESULTS: We discuss criteria for considering if a rapid advice guideline is appropriate and feasible and outline the roles of various contributors across the phases of development. Further, we describe the methods and steps involved in performing rapid reviews, which are more fluid and iterative than for a standard guideline process. In general, rapid advice guidelines involve a shorter timeline, narrower scope, and the use of abbreviated methods for the evidence review. CONCLUSION: Important differences exist between developing a standard guideline and a rapid advice guideline. However, the core principles for WHO guidelines apply to rapid advice guidelines including minimizing bias, applying transparent processes and the use of explicit methods. The World Health Organization (WHO) produces global guidelines for the 193 Member States of the United Nations. WHO defines guidelines broadly, as ''any document developed by the WHO containing recommendations for clinical practice or public health. A recommendation tells the intended end user of the guideline what he or she can or should do in specific situations to achieve the best health outcomes possible individually or collectively'' [1] . Each guideline developed by WHO (or any organization) needs to best fit the intended purpose and meet the end users' needs, and this determines the methods, resources, and timeline for development, dissemination, and implementation. The WHO Handbook for Guideline Development, 2nd Edition (2014) (''WHO Handbook''), outlines four main types of guidelines: standard guidelines, consolidated guidelines, interim guidelines, and guidelines developed in response to a public health emergency or urgent need such as a natural disaster, warfare, biologic or chemical exposures, or an unforeseen disease epidemic (Box 1) [1] . In the context of such emergencies, WHO must at times provide global leadership within hours to days. Such The principles underlying rapid advice guidelines are the same as for standard guidelines. All steps should be tailored to the situation, and some can be appropriately abbreviated from standard processes. This paper describes the considerations that are relevant to deciding if a rapid advice guideline should be developed in the context of a public health emergency and outlines the processes and methods for developing such guidelines. To date, WHO has published two rapid advice guidelines based on this approach, both developed in 2014 in the context of the filovirus (Ebola) outbreak. It is possible to apply rapid review and rapid advice guidelines methods to complex public health interventions in emergency situations where the end users may be very diverse. What is the implication and what should change now? WHO has a transparent process for producing evidence-based recommendations in the context of public health emergencies. Further research is needed comparing rapid advice guidelines to standard guidelines with regard to utility, implementation, and health impact. guidelines are termed as ''emergency (rapid response) guidelines'' and processes and methods for producing such guidelines are currently under development at WHO [1] . However, if a public health emergency continues and as response efforts evolve into recovery and rebuilding, guidelines are needed that are developed using more rigorous methods and generally with a somewhat longer timeline: perhaps 1 to 3 months. These are termed ''rapid advice guidelines'' and are the focus of this paper. Rapid advice guidelines must meet minimum standards, the recommendations must be based on evidence and they are subject to an internal quality control and assurance process. The process and methods used for their development may be modified from those of standard guidelines, to meet the accelerated timeline necessitated by Member States' needs in the context of the emergency [1] . The objective of this paper is to describe the criteria that WHO staff use to assess the need for developing a rapid advice guideline and to outline the steps and methods for developing such a guideline in the context of a public health emergency. This article is a synopsis of more detailed methods for developing rapid advice guidelines, which can be found in the WHO Handbook on Guideline Development (2nd Edition, 2014) [1] (see Chapter 11) . Our purpose is to advance transparency of WHOs guideline development process and to provide external organizations with a description of an approach that might be useful and applicable to their contexts. To date, WHO has published two rapid advice guidelines based on this approach, both developed in 2014 in the context of the filovirus (Ebola) outbreak: one examining hand hygiene and the use of chlorine; [2] the other on the effectiveness of various components of personal protective equipment for healthcare workers [3] . Although we began development of methods specific to WHO rapid advice guidelines before the Ebola crisis emerged, it is a compelling example of how we were able to apply these newly developed methods and finalize them based on lessons learned in the context of Ebola (see Box 2) . WHO has issued guidelines labeled as ''rapid'' in the past. A 2007 publication describes the production of a guideline where most all of the standard steps and methods were executed within an 8-to 10-week time frame [5] . Since that publication, the term ''rapid'' has been used in the title of several WHO guidelines. However, none of these guidelines was, in fact, produced rapidly, and none reported using unique or modified approaches: rather they described standard approaches in the context of efforts to produce the guideline rapidly. The current work builds on that prior work, focusing on when standard guideline methods can and should be abbreviated to meet the needs of Member States in a timely manner. This guidance on rapid advice guideline development is based upon an existing rapid review approach, which was modified to meet WHOs needs and to allow integration with the organization's existing approach to developing standard guidelines. The rapid review approach [6] and typology [7] were developed by the Knowledge Synthesis Group at the Ottawa Hospital Research Institute. This approach consists of an eight-step process based upon widely accepted systematic review methods, particularly those of the Cochrane Collaboration [6] . This approach has been used to develop rapid reviews for a variety of types of decision makers and has undergone modifications as needed to optimize the approach. In addition, this guidance on how to develop rapid advice guidelines was informed by discussions with WHO staff involved in emergency response (n 5 6), including staff from the Global Influenza Programme, Department of Food Safety and Zoonoses, Global TB programme, HIV Department, Emergency Risk Management Department, and the WHO Headquarters Library. The primary purpose of these informal dialogues was to become more familiar with the current WHO guideline process and to understand staff roles, experiences, and needs with regard to development of rapid advice guidelines. This was not considered research as there was no formal structure to the discussions and no data collection, analysis, or reporting. These discussions therefore did not require research ethics approval. The first step when planning the development of any guideline, including a rapid advice guideline, is to search for relevant, high-quality existing guidelines. If such a guideline already exists, it may be adopted or adapted by WHO staff at headquarters or in regional or country-level offices or at the subnational and facility level. However, if no relevant guidelines are identified, there are a number of important considerations when deciding to develop a rapid advice guideline vs. a standard guideline or to defer development of a guideline altogether. 3.1. What is the type of emergency and the risk to public health? The first step is to examine the public health event that is driving the request for a rapid advice guideline. Emergencies may be classified as natural, technological, or conflict related and may be of sudden onset (e.g., earthquakes, tsunamis, chemical crises) or more gradual onset (e.g., Box 1 Types of WHO guidelines [1] There are four main types of guidelines produced by WHO that comprise a broad spectrum of products that vary mainly in terms of the following features: -Purpose; -Scope; -The point in time at which the guideline is being developed relative to the life span of an intervention; -The organizations or entities developing the guideline; -The presence in the guideline of new vs. previously published recommendations; and -The timeline. deteriorating situations in armed conflict, progressive disease outbreaks, drought, or food insecurities). All types of emergencies can evolve into protracted situations. WHO and the Member States of the United Nations use the Rapid risk assessment of acute public health events manual to assess ''any outbreak or other rapidly evolving situation that may have negative consequences for human health and requires immediate assessment and action'' [8] . Risk is characterized by level and is based on broad descriptive definitions of likelihood and consequences, represented in the form of risk matrices. The WHO Emergency Response Framework describes WHOs roles and responsibilities between the initial alert of an event and its subsequent classification [9] . WHO categorizes emergencies from grade 1 (those with minimal expected public health consequences) to grade 3 (those involving events in one or more countries and having significant public health consequences that call for a substantial regional and/or international response). WHO staff may consider producing a rapid advice guideline in the face of either a new situation (e.g., a new strain of influenza, the Middle East respiratory syndrome coronavirus, or an earthquake) or an event encountered previously but causing problems in a different context (e.g., a change in disease pattern such as the Ebola virus disease outbreak in West Africa in 2013 or a prolonged armed conflict compounded by a disease outbreak). If the event is not novel, high-quality relevant guidelines may already exist and a new guideline may not be needed. Guidelines are indicated when there is uncertainty about what to do in a specific situation. WHO staff may be uncertain about what advice to provide or there may be uncertainty in the field, with different stakeholders having different viewpoints and approaches. In determining if a rapid advice guideline is appropriate, the key question is how quickly the uncertainty needs to be dealt with. Rapid advice guidelines can generally be developed within 1 to 3 months. If an event is likely to persist beyond 6 months, a rapid advice guideline may not be optimal and a standard guideline may be the best Box 2 An example of a rapid advice guideline developed Personal protective equipment in the context of filovirus disease outbreak response: Rapid advice guideline (2014). World Health Organization. Context -A public health emergency of international concern. Issue -Healthcare workers caring for individuals with Ebola were at an increased risk of contracting Ebola virus disease during the outbreak in West Africa starting in 2013. -There was uncertainty in the field as to the most effective types of personal protective equipment (PPE). Development of a WHO rapid advice guideline -A rapid review (RR) was conducted over 7 weeks to inform the recommendations [4] . -Initially, the RR focused on the comparative effectiveness and disadvantages of PPE (gloves, gowns, and face protection) for healthcare workers working with Ebola patients. However, only noncomparative studies were identified. -Concurrent with the RR, a survey of values and preferences was administered to expatriated healthcare workers over a 3-week period, which helped to inform recommendations. -The noncomparative data from the RR, the survey data, and information from experts in virology and bloodborne pathogens and materials science formed the basis for the recommendations which were formulated at an expert meeting. Significance -Produced over a 12-week time frame [3] , this marked the first rapid advice guideline produced by WHO following the approaches outlined herein. Abbreviation: WHO, World Health Organization. approach. On the other hand, if the emergency is likely to be transient, then existing guidelines should be reviewed for applicability, with the production of emergency (rapid response) guidelines as appropriate. It is important to weigh the impact of developing recommendations using standard processes and timelines vs. producing a guideline that may be prone to serious limitations under an accelerated timeline. 3.5. Will the rapid advice guideline be rapidly implemented? Rapid advice guidelines should only be developed if a mechanism is either already in place or likely will be in place for disseminating and implementing the recommendations in the guideline in the context of the emergency. Various factors need to be carefully considered: the existence of functioning health systems; adequate health workforce; necessary infrastructure; the acceptability of the proposed intervention; the training requirements; and resource availability. The basic steps for developing a rapid advice guideline are depicted in Tables 1 and 2 and are generally the same as those that apply to standard guidelines. There are, however, some differences and additional considerations when developing a rapid advice guideline. Once the relevant WHO technical unit determines that a guideline is needed, the unit contacts the Guidelines Review Committee Secretariat whose remit is to support the WHO Guidelines Review Committee, which is responsible for setting the standards, developing the methods, and assuring the quality of all guidelines issued by WHO [1] . The Secretariat will assist the technical unit in deciding if the topic is suitable for a rapid advice or other type of guideline and will provide technical support if the unit moves ahead with guideline development. When developing a rapid advice guideline, four key groups need to be established quickly. First is the internal WHO Steering Group whose primary responsibility is to oversee the rapid advice guideline development process. Second, the review team will produce a rapid, yet comprehensive and objective synthesis of the evidence to inform each recommendation. A methodologist with expertise in guideline development processes and methods is also identified early in the development process. Third, the external review group contributes diverse and real-world perspectives at the peer-review stage. Fourth, the Steering Group assembles the Guideline Development Group, which provides input on the scope and content of the rapid advice guideline, and is primarily responsible for formulating the recommendations. The Guideline Development Group must include a broad range of relevant clinical and public health technical and programmatic expertise, as well as representation from key stakeholders such as persons who will be affected by the recommendations in the guideline. The Guideline Development Group must have geographic representation from all WHO regions and must be gender balanced to the extent possible ( Fig. 1) . It is critical to include individuals with expertise in ethical, social, and legal dilemmas on the Guideline Development Group, as well as expertise in issues related to equity, gender, and human rights. Although these issues may be considered by some to be peripheral to the urgent health problem being addressed (e.g., an outbreak of a disease), critical human rights issues often emerge in the context of a public health emergency, and they must be addressed in the initial stages of a response. Once the need for a rapid advice guideline has been established, the WHO Steering Group continues to redefine the scope of the guideline and develop key questions in population, interventions, comparators, outcomes (PICO) format. A rapid advice guideline will most likely provide recommendations on the benefits and harms of interventions. However, recommendations on diagnostic tests, prognosis, and risk factors may also be needed. With the assistance of an experienced information specialist, a scoping exercise should be conducted quickly to provide a general sense of the depth of the relevant literature. This is not a systematic search of all potential sources, but rather a focused search for the best available, relevant literature, including high-quality systematic reviews and key primary studies. The resources most applicable to the topic should be examined briefly (e.g., MEDLINE, The Cochrane Library, Scopus, etc.) in addition to looking for any information or guidance published by WHO in the early stages of the public health emergency. This scoping exercise, including synthesis of the evidence retrieved, should take no longer than 1 or 2 days, and a brief summary of the results should be prepared. A detailed planning proposal akin to a review protocol should be prepared for all guidelines, including rapid advice guidelines. At WHO, all planning proposals are reviewed by the Guidelines Review Committee, and in the context of a rapid advice guideline, the primary issue for the Guidelines Review Committee is to determine if there is adequate justification for applying an accelerated and abbreviated process. The planning proposal for rapid advice guidelines has the same content, level of detail, and format as for standard guidelines, describing the planned processes and procedures, the results of the scoping review, the methods for the rapid review, and the approach for translating the evidence into recommendations. The planning proposal serves as a point of reference for all contributors, and therefore, it must be detailed and kept up to date, even when operating under a compressed timeline. This is particularly important as contributors to the guideline may change as WHO staff members are deployed to the field during the guideline development process. As described below, the rapid review process is often more fluid and iterative than that of a standard systematic review, and thus, the planning proposal is a living document, amended as needed, including the rationale for any changes. Complete and accurate documentation ensures transparency and greatly facilitates the drafting of the final guideline document. Performing rapid reviews and developing summaries of the evidence 5.1. What are rapid reviews? When rapid advice guidelines are deemed necessary, conducting a systematic review de novo may not be feasible. Rapid reviews have emerged as a streamlined approach to identifying and synthesizing evidence, typically for the purpose of assisting expeditious decision making by state and local governments or by healthcare providers. For the purposes of this guidance, we define ''rapid review'' as a type of evidence review that is produced using accelerated and/or modified systematic review methods [6] . The core principles of evidence searching and retrieval for standard systematic reviews apply to rapid reviews, including thoughtful scoping and formulation of the review questions, transparency, reproducible methods, careful assessment of the quality of the information incorporated into the review, efforts to minimize bias at every stage, and the clear presentation of information focused on the intended users' needs. However, there are important differences: the rapid review may have a more limited scope and fewer outcomes of interest, more restricted search criteria, looks to existing high-quality systematic reviews as the first line of evidence, involves a more targeted and iterative procedure for screening The technical unit must determine if a rapid advice guideline is needed or if a standard or interim guideline would be more appropriate. Discuss the process with GRC Secretariat and with other WHO staff with experience developing guidelines. The planned guideline is discussed with the Secretariat when it first becomes a possibility. Form the Steering Group. All relevant departments at WHO headquarters and in the regional offices must be involved. Identify sufficient resources. Determine the timeline. Draft the scope of the guideline. Begin preparing the planning proposal. The literature is scoped through a brief review. The guideline's scope must be narrow and feasible. Identify potential members of the GDG and the chair. Issue invitations early; involve the GDG in determining the scope and key questions. Obtain DOIs and manage any COIs among potential GDG members. The process for rapid advice guidelines and standard guidelines is identical. Steering Group and the Guideline Development Group (GDG) Formulate key questions in PICO format. Prioritize outcomes. Key questions (in PICO format) include only those of the highest priority and must be focused and narrow. Background questions are not addressed in a rapid advice guideline. Finalize the guideline planning proposal. The process is the same as for a standard guideline. Review and approve the planning proposal. The GRC uses an accelerated process for review and disposition. Abbreviations: COI, conflict of interest; DOI, declaration of interest; PICO, population, intervention, comparator and outcome; WHO, World Health Organization. the literature and for data analysis and synthesis, places less emphasis on meta-analyses, and involves a concise and abbreviated report. In addition, in a rapid review, the search process is more iterative and hierarchical, depending on the findings at each step: the types of publication and study designs included and the bibliographic databases searched may change as the evidence is explored. Other efficiencies may be achieved by, for example, adding more resources so that reviewers can work in parallel. Types of reviews that underpin rapid advice guidelines may be categorized into two basic types: a standard systematic review performed rapidly or a rapid review involving a variety of abbreviated methods, which may include only existing systematic reviews; primary studies and existing systematic reviews; or only primary studies (Table 3) Depending on the nature of the question being asked, the purpose of the rapid review, and the magnitude of the literature on the topic, various types of evidence may be targeted. In most cases, the emphasis will be placed on locating and summarizing evidence from relevant and high-quality ''offthe-shelf'' systematic reviews or guidelines. In the absence of such systematic reviews, high-quality and/or recent primary studies may be included. Landmark papers may be included for reference, and high-quality quasiexperimental or observational studies may be considered, depending on the key question and the volume of the available evidence. Usually, no more than two to three of the most relevant databases are searched (e.g., MEDLINE, The Cochrane Library, EMBASE, Scopus). However, depending on the review topic and access to research databases, additional databases including topic specific and regional databases may be examined (e.g., PsychINFO, CINAHL, ERIC, African Index Medicus, International Clinical Trial Registry Platform, ClinicalTrials.gov). A WHO information speciality should be involved in the selection of the priority information sources, as regional databases and local sources may be the richest source of relevant information. In a standard systematic review, the aim is to maximize both recall, which is the ability to identify all relevant articles (sensitivity) and precision, which is the ability to The contractor needs to be identified from the outset and involved in the scoping and development of key questions: they can advise on what is feasible in the given time frame. Evaluate evidence quality for each important outcome, using GRADE as appropriate. The process is the same as for a standard guideline. Convene a meeting of the GDG. Meeting place and participants need to be identified at the beginning of the development process. The meeting has a similar format and agenda as for the development of a standard guideline. Guideline Development Group (GDG) Formulate recommendations using the GRADE framework. The general methods are the same as for a standard guideline. The evidence may be sparse, so other factors that inform the recommendations must be transparent and based on indirect evidence when possible, and on equity, human rights and gender considerations. exclude nonrelevant articles (specificity). However, for a rapid review, the aim may be to maximize precision rather than recall. Several common eligibility restrictions should be considered to optimally balance recall and precision (Box 3) [1] . Potential restrictions should be discussed among WHO Steering Group members and with the review team information specialist. Search strategies for a rapid review will generally have language restrictions because translation is time consuming. The languages of inclusion should be carefully selected based on the guideline topic. For example, a rapid review on personal protective equipment for health workers in Ebola treatment centers [4] , engendered by the Ebola virus disease outbreak that became widespread across parts of West Africa in 2014, included only literature in English and French owing to the geographic distribution of the outbreak and the opinion of experts that most of the relevant literature was in those two languages. Citations in nonselected languages are generally included during the citation screening phase but may be excluded from further analyses if the full text is difficult to access or insufficient time or resources are available for translation. Search terms should include both medical subject headings (MeSH) and text words. Validated search filters may be useful (see Chapter 8, WHO Handbook [1] ), such as those related to study type and design (e.g., randomized controlled trial, systematic review, or meta-analysis). The draft search strategy must be reviewed by at least one other member of the rapid review team, one or more content experts, and a WHO information specialist. A limited search for gray literature should be considered (e.g., relevant data may be quickly identified and retrieved from the web sites of relevant organizations). The search approach and restrictions used, and their rationale and potential limitations should be reported in the planning proposal, the review report, and the guideline document. A list of potentially relevant citations identified during the search but excluded from the analysis due to language restrictions or other reasons should be included as an appendix in the rapid review report. In the context of a new situation or event, the best (and perhaps only) data might come from the analysis of emerging information in real time. In the Ebola virus disease outbreak in West Africa in 2014, essentially, no relevant data were obtained through a rapid review of the published literature comparing various types of personal protective equipment in the context of Ebola or related viruses [4] . Therefore, a survey of repatriated healthcare workers was rapidly implemented to gather information on experiences with various types of personal protective equipment [10] . If time permits, the reference lists of all included studies should be scanned for additional relevant studies to ensure that key publications have not been overlooked. Standard systematic review methods apply to the process of screening the records retrieved via the searches. Records should be imported into reference management software to facilitate record management and citation screening. Study selection involves a two-step process. First, either two people independently screen titles and abstracts of all potentially relevant records or one person reviews all titles and abstracts, whereas the second reviewer examines only the citations excluded by the first. Second, two reviewers examine the full-text publications to determine their eligibility. As for a standard systematic review, consensus on the included studies should be achieved, with involvement of a third reviewer if necessary. To keep the scope of a rapid review within the bounds dictated by timelines and resources, initially, the evidence is often limited to that found in systematic reviews. A decision to include primary studies must be justified in the planning proposal and reflected in the timelines and budget. Further restrictions (e.g., by outcomes or study quality) may be considered to accommodate the inclusion of primary studies. Records that are not available electronically are generally excluded because the timeline of a rapid advice guideline is not compatible with the delays involved in interlibrary loans. Even if the full text cannot be obtained or translated, the abstract may provide valuable information, particularly when evidence is sparse. Once the included studies are finalized for each critical outcome, outcome data can then be extracted, including key study demographics, effect estimates (e.g., odds ratios, mean differences, or summary effect [i.e., a metaanalysis]), and their corresponding confidence intervals. A standard extraction form should be developed and pilot tested to facilitate accurate data collection. Usually, one reviewer extracts data, and a second verifies all extracted data. If this is not feasible, a random sample of at least 10% of the included studies should be independently checked to provide some measure of quality assurance. The rapid review team will finalize the data analysis plan in consultation with the WHO Steering Group. Quantitative syntheses of primary studies (i.e., meta-analyses) may not be feasible for rapid reviews unless time and resources permit; however, the results of previously published metaanalyses should be reported. Fig. 2 provides details of the various steps and decisions involved in selecting the type of evidence and the approach to data synthesis (see Chapter 11, WHO Handbook [1]). The risk of bias should be assessed for each included study to facilitate appropriate interpretation of the review findings. For rapid reviews particularly, the assessment of the risk of bias may be used to select the studies included in the review, once initial criteria based on study design have been applied. The quality of the body of evidence for each outcome that is critical for decision making should generally be assessed using the Grading of Recommendations Assessment, Sources -Usually, search no more than two or three key bibliographic databases. -If time and resources permit, additional resources may be added. Language -Language restrictions are frequently applied, as translation is time consuming and resource intensive. -Limitations by language of publication need to be assessed for each topic, with consideration given to the distribution of the disease or condition being addressed and the likely languages of the relevant publications. Accessible studies -Publication status is limited to full text only (abstracts are not usually included). -To maximize efficiency, articles should be electronically available through e-journal subscriptions available to the rapid review team. -Articles should be purchased directly from a journal only under special circumstances, namely when the paper is deemed essential and is not available through other means. Gray literature -The utility of the gray literature is assessed for each topic. -Web sites of relevant organizations may be examined, depending on the subject under review. Year (search dates) -Publication dates are limited (e.g., only the most recent decade is searched). -When applying a year limit, a rationale for the time frame must be provided. -Restrictions may be placed on the geographical locations of the included studies. -A rationale should be provided to explain why citations from certain regions, rather than from the global literature, are targeted. Development and Evaluation (GRADE) framework [11] . The focus is on health outcomes and not on intermediate, surrogate, or other types of outcomes. However, exceptions may be made when data are sparse, and decisions may need to be based on indirect evidence, including intermediate outcomes. Rapid reviews often necessitate the inclusion of existing systematic reviews over primary studies. However, when using an existing systematic review that did not use the GRADE framework to assess the quality of the body of evidence (or which does not supply all of the necessary information for this assessment), it may not be feasible to examine the individual studies included in the review to assess their risk of bias and to develop GRADE profiles de novo. In this case, ROBIS [12] , a tool for assessing the risk of bias in systematic reviews (rather than in primary studies) and where appropriate the relevance of a review to the research question at hand, could be applied. Further, to determine a review's quality, A Measurement Tool to Assess systematic Reviews [13] could also be used. Although limited in the ability to assess quality in terms of certainty of the effect estimates, this will help to identify areas of potential concern to help judge overall risk of bias and the quality of conduct across included reviews. If only primary studies are identified, then it will be important to assess risk of bias at the individual study level (See Box 4) applying the GRADE framework. The rapid review report should transparently and succinctly summarize the methods used and the results of the review. Suggested components of the rapid review report are listed in Box 4. The rapid review methods should be reported at a level of detail that will allow them to be replicated by interested organizations and readers. A PRISMA flow diagram [14] gives the reader an overview of the rapid review process and a snapshot of the evidence identified. All rapid reviews should include a narrative summary of the evidence, generally organized around the PICO framework. A brief section on the gaps in the evidence and future research needs may be very useful, particularly when data are sparse. A written disclosure should be provided that the rapid review is not intended to be a gold standard systematic review and that its results should be interpreted with caution and viewed within a specific context. The Steering Group needs to plan early for the Guideline Development Group meeting where recommendations will be formulated. Recommendations can be developed via a virtual meeting, although in-person meetings are preferred, even in the context of a rapid advice guideline. The GRADE approach for formulating recommendations should be followed when developing rapid advice guidelines (see Chapter 10dWHO Handbook [1] ). It will seldom be feasible to collect primary data or to perform a review of the resource implications of the intervention or of the values and preferences surrounding the outcomes of interest. However, data that can be readily obtained should be collected (e.g., the cost of gloves in the 2014 guideline on personal protective equipment in the context of Ebola virus disease) [3] . Implementation and the importance of context should also be considered when developing a rapid advice guideline as most research evidence was likely generated in settings and populations that differ from that of the public health emergency at hand. Thus, the degree to which such evidence can be directly applied to the current context may be limited. It is important to consider how contextual factors can modify the benefits and harms of an intervention and how various barriers and facilitators can affect implementation and impact. For a rapid review that relies heavily on evidence from systematic reviews, the synthesis should be tailored to the local context for the emergency throughout all stages of the guideline development process. The process and resources needed to draft the final rapid advice guideline document are the same as for standard guidelines, and a writer should be identified early and involved throughout the development process. It is particularly important to describe how the rapid advice guideline differs from a standard guideline, and the potential biases that may have been introduced. In addition, the shelf life of the document should be clearly indicated; for example, if the rapid advice guideline constitutes interim guidance because new information is anticipated in the foreseeable future, this should be clearly indicated to the user. A draft of the rapid advice guideline draft should be peer reviewed by key individuals, both internal and external to WHO. Three to six potential peer reviewers should be identified early and their interest, availability, and commitment to a quick turn-around time (e.g., 48e72 hours) discussed. Governmental or nongovernmental organizations that are involved in the public health emergency should also be asked to review the draft document to promote engagement and buy-in during dissemination and implementation and to raise issues and concerns before publication. At an absolute minimum, all relevant departments at WHO and in the regional offices must be given the opportunity to provide substantive input into the final document. Publication of the final rapid advice guideline involves the same steps as for a standard guideline. Electronic means will usually be used for initial dissemination, followed by print circulation as required in the local context. WHO must produce high-quality, evidence-informed guidelines in the context of public health emergencies when there are no existing guidelines for Member States to implement. We have outlined the processes and methods by which WHO can produce rapid advice guidelines in this context. The development of a rapid advice guideline differs in important ways from that of a WHO standard guideline. A rapid advice guideline has a very narrow scope to make development feasible within the given time frame. Moreover, WHO staff and external experts need to be identified and engaged early in the guideline development process, and the Guidelines Review Committee Secretariat should be contacted to put in place the required expedited processes and to provide technical support. Rapid review methods may differ from those of a traditional systematic review, including constraints in searching bibliographic databases and other sources of information; the need for a more fluid and iterative approach to establishing study inclusion/exclusion criteria, data extraction, and evidence synthesis; and the abbreviated nature of the review report. These differences, in turn, may affect the credibility of the review and the validity of the review's conclusions. Given that interest in rapid reviews has increased and there is great variability in the approaches and level of reporting [15e19], future research needs to address how rapid reviews compare with standard systematic reviews in terms of bias and credibility, with further guidance developed on when and how to conduct a rapid review. Rapid reviews have become an area of new methodological development for several health research organizations. The US Agency for Healthcare Research and Quality has established a rapid reviews workgroup [20] . Cochrane, the world's largest producer of high-quality systematic reviews of effectiveness, recently established the Cochrane Methods Rapid Reviews Group [21] . Further, due to the increased interest of public authorities and clinicians, the Guidelines-International-Network established a working group dedicated to the methods for developing guidelines in an accelerated time frame [22] . Few data exist on the ways in which rapid advice guidelines are developed and implemented, how they differ from standard guideline development methods, and the impact of rapid advice guidelines on health outcomes. Nevertheless, the core principles and standards for WHO guidelines apply: minimize bias; apply transparent processes and explicit, reproducible methods; acknowledge potential limitations; and attend to the target audience's needs and to the interests of the individuals and populations affected by the recommendations. Applying these principles and meeting these standards in the face of an emergency involves trade-offs, as well as expertise in both guideline development methods and the guideline topic. Further, guideline developers at WHO need to commit to updating these guidelines in a timely manner when new data become available. When warranted, rapid advice guidelines need to be converted to standard ones so that WHO recommendations are robust and the organization is prepared for continuing public health emergencies or for recurrent events. Introduction -Brief description of the rationale for the rapid review and of the context for the guideline. -Duration of the rapid review process (with accompanying dates). -Indication that this is a rapid review and should be interpreted in that light. Methods WHO Library Cataloguing-in-Publication Data Hand hygiene in health care in the context of filovirus disease outbreak response World Health Organization. Personal protective equipment in the context of filovirus disease outbreak response: rapid advice guideline Effectiveness of personal protective equipment for healthcare workers caring for patients with filovirus disease: a rapid review Transparent development of the WHO rapid advice guidelines Evidence summaries: the evolution of a rapid review approach Rapid Reviews: The Ottawa Experience. Presentation to: the World Health Organization (WHO) Guideline Review Committee Secretariat Rapid risk assessment of acute public health events. Geneva: World Health Organization World Health Organization The incorporation of healthcare workers' values and preferences on personal protective equipment into a rapid advice guideline on Ebola. Accepted abstract Grade Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations ROBIS: A new tool to assess risk of bias in systematic reviews was developed AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement Rapid reviews versus full systematic reviews: an inventory of current methods and practice in health technology assessment Expediting systematic reviews: methods and implications of rapid reviews What is a rapid review? A methodological exploration of rapid reviews in Health Technology Assessments Advancing knowledge of rapid reviews: an analysis of results, conclusions and recommendations from published review articles examining rapid reviews Rapid review programs to support health care and policy decision making: a descriptive analysis of processes and methods Available at: www.g-i-n.net/working-groups/ accelerated-guideline-development/accelerated-guideline-development-overview The authors are grateful to Dr. Dianna Wolfe, Senior Research Associate at the Ottawa Hospital Research Institute, for providing copy edit support on the submitted manuscript.