key: cord-0993587-wcm3a6ro
authors: Schünemann, Holger J; Hill, Suzanne R; Kakad, Meetali; Vist, Gunn E; Bellamy, Richard; Stockman, Lauren; Wisløff, Torbjørn Fosen; Mar, Chris Del; Hayden, Frederick; Uyeki, Timothy M; Farrar, Jeremy; Yazdanpanah, Yazdan; Zucker, Howard; Beigel, John; Chotpitayasunondh, Tawee; Hien, Tran Tinh; Özbay, Bülent; Sugaya, Norio; Oxman, Andrew D
title: Transparent Development of the WHO Rapid Advice Guidelines
date: 2007-05-29
journal: PLoS Med
DOI: 10.1371/journal.pmed.0040119
sha: 8275311de465e4fdbb25ebddf76302cd4603c36c
doc_id: 993587
cord_uid: wcm3a6ro

Emerging health problems require rapid advice. We describe the development and pilot testing of a systematic, transparent approach used by the World Health Organization (WHO) to develop rapid advice guidelines in response to requests from member states confronted with uncertainty about the pharmacological management of avian influenza A (H5N1) virus infection. We first searched for systematic reviews of randomized trials of treatment and prevention of seasonal influenza and for non-trial evidence on H5N1 infection, including case reports and animal and in vitro studies. A panel of clinical experts, clinicians with experience in treating patients with H5N1, influenza researchers, and methodologists was convened for a two-day meeting. Panel members reviewed the evidence prior to the meeting and agreed on the process. It took one month to put together a team to prepare the evidence profiles (i.e., summaries of the evidence on important clinical and policy questions), and it took the team only five weeks to prepare and revise the evidence profiles and to prepare draft guidelines prior to the panel meeting. A draft manuscript for publication was prepared within 10 days following the panel meeting. Strengths of the process include its transparency and the short amount of time used to prepare these WHO guidelines. The process could be improved by shortening the time required to commission evidence profiles. Further development is needed to facilitate stakeholder involvement, and evaluate and ensure the guideline's usefulness.

May 2007 | Volume 4 | Issue 5 | e119 C linical practice guidelines generally, and some WHO guidelines specifi cally, have been criticized for not being based on the best available evidence, for being exposed to undue infl uence by industry and experts who participate in guideline panels, and for not adhering to guidelines for preparing guidelines [1] [2] [3] [4] [5] [6] [7] . Guidance that is not informed by the best available evidence or by statements that the available evidence is of low quality can harm patients, waste limited resources, and hinder research to address important uncertainties [8] .

While there is broad agreement that rigorous and transparent methods should be used [9] [10] [11] [12] , rigorous development of guidelines can take two years or more [13, 14] . This timeframe is not practical for providing rapid advice, for example for emerging infectious diseases such as avian infl uenza (H5N1 infection) or severe acute respiratory syndrome (SARS). Indeed, one of the most frequently cited weaknesses in guideline development is the length of time that it takes to develop a guideline [15] . Organizations including the National Centre for Health and Clinical Excellence in the United Kingdom and the National Institutes of Health in the United States are investigating ways of streamlining guidelines development processes [16, 17] .

The Health in Action section is a forum for individuals or organizations to highlight their innovative approaches to a particular health problem.

Emerging health problems require rapid advice. We describe the development and pilot testing of a systematic, transparent approach used by the World Health Organization (WHO) to develop rapid advice guidelines in response to requests from member states confronted with uncertainty about the pharmacological management of avian infl uenza A (H5N1) virus infection. We fi rst searched for systematic reviews of randomized trials of treatment and prevention of seasonal infl uenza and for nontrial evidence on H5N1 infection, including case reports and animal and in vitro studies. A panel of clinical experts, clinicians with experience in treating patients with H5N1, infl uenza researchers, and methodologists was convened for a two-day meeting. Panel members reviewed the evidence prior to the meeting and agreed on the process. It took one month to put together a team to prepare the evidence profi les (i.e., summaries of the evidence on important clinical and policy questions), and it took the team only fi ve weeks to prepare and revise the evidence profi les and to prepare draft guidelines prior to the panel meeting. A draft manuscript for publication was prepared within 10 days following the panel meeting. Strengths of the process include its transparency and the short amount of time used to prepare these WHO guidelines. The process could be improved by shortening the time required to commission evidence profi les. Further development is needed to facilitate stakeholder involvement, and evaluate and ensure the guideline's usefulness.

The WHO Advisory Committee for Human Research, an independent committee appointed by the Director-General of the WHO, evaluated existing WHO and other guidance processes and suggested ways to improve WHO's methods [18, 19] . Consistent with these recommendations, a new model for developing WHO rapid advice guidelines was designed and tested through the development of guidelines for the pharmacological management of avian infl uenza A (H5N1) virus infection [20, 21] . Because the approach to developing these rapid advice guidelines was novel for WHO, we describe the methods, the strengths of the approach, and ways in which this approach should be further developed.

In January 2006, the WHO decided to convene a rapid advice guidelines panel for the pharmacological management of H5N1 patients in response to requests for advice from frontline clinicians and public health professionals managing H5N1 infections. The key steps and timeline for developing the guidelines are summarized in Box 1.

Group selection and composition. In selecting members for the panel, we wanted to include several important stakeholders: clinical, methodological, and basic science experts and member country representatives, including low-and middle-income countries. We used the WHO's international network of response teams for viral pandemics and searched the medical literature for experts on H5N1 infection to identify panel members. We asked methodologists with experience in applying the Grading Recommendations Assessment, Development and Evaluation (GRADE) approach that was offi cially adopted by WHO to participate in the creation of evidence profi les and the guideline development process. Thirteen voting panel members supplemented by WHO experts participated in the panel meeting. We followed a thorough process to declare confl icts of interest (see Text S1 under Supporting Information).

Formulating questions and rating the importance of outcomes. The original questions were identifi ed by clinicians managing patients with H5N1 infections and refi ned by panel members. For each question an evidence profi le was prepared using the GRADE approach ( Figure 1 ) [22] . The GRADE approach required the identifi cation of relevant outcomes to be included in each evidence profi le before developing the evidence profi les. Two reviewers identifi ed potentially important outcomes and this list was circulated to the panel chair, WHO staff, and the scientifi c reviewers by email for independent scoring of the relative importance of each outcome and identifi cation of additional potentially important outcomes. Outcomes were rated on a scale from 1-9; a rating of 7-9 indicated that the outcome was critical for a decision or recommendation, 4-6 indicated it was important, and 1-3 indicated it was not important. The evidence profi les included only critical or important outcomes based on the mean value of the ratings by the panel members.

To obtain consumer input, the Cochrane Consumer Network was also invited to provide feedback through their electronic discussion list. We received only four responses, but despite reported diffi culties with the rating, the relative importance of the outcomes did not differ importantly from those of the panel and no additional outcomes were identifi ed.

Preparation of evidence profi les. An independent review team searched for systematic reviews and recent randomized trials (published in 2005 or 2006) for the treatment and chemoprophylaxis of any infl uenza virus infection; and case series, animal studies, and in vitro studies for the treatment or chemoprophylaxis of H5N1 infection (from 1966). The team prepared evidence profi les using the GRADE profi ler software (v1.12, http:⁄⁄www.gradeworkinggroup.org) and proposed quality ratings according to the criteria in Box 2. The quality of outcomes measured in each animal study was judged based on whether (1) the pathogenicity of H5N1 virus was tested in the model (e.g., mortality), (2) statistical methods were adequate, and (3) a signifi cant effect was demonstrated.

A summary of the fi ndings for each question, including both trial evidence for non-H5N1 infl uenza and the available evidence for H5N1 infection, was prepared for each question ( Figure  2 ) [21] . Panel meeting. Guideline group members and an independent expert received evidence profi les about two weeks prior to the meeting, including information about the applied methods. They were asked to identify any important missing evidence.

A draft of the guidelines was prepared by the WHO secretariat and the panel chair prior to the meeting. For the actual meeting, the agreed process rules were that: (1) Additional evidence would only be allowed at the meeting if it had been omitted from the evidence summaries, or was new and critical for decision making; (2) The GRADE approach would be used to grade the quality of evidence and the strength of recommendations [10, 21] ; (3) Recommendations would be based on a consensus of the panel and voting would be used if agreement could not be reached; (4) All panel members would be asked to consider their own and other confl icts during the discussion and decision making and to abstain from discussion and voting if necessary (see Supporting Information); (5) Subsequent interaction and discussion would take place through email but recommendations would not be changed after the meeting, except for minor wording changes or correction of factual errors.

Deliberation regarding the balance of benefi ts, harms, and costs. For each intervention considered, the panel formulated a recommendation based on the panel members' judgments regarding the balance between the benefi ts, harms (adverse effects), burdens (e.g., taking medication daily), costs, and values and preferences (the desirability or preference that individuals exhibit for a particular outcome) of the intervention (see Supporting Information Text S1 for information on cost).

Recommendations were classifi ed as "strong" or "weak." The panel was informed that strong recommendations should be interpreted as: (1) Most individuals should receive the intervention; (2) Most wellinformed individuals would want the recommended course of action and only a small proportion would not; (3) The intervention could unequivocally be used in policy making.

Weak recommendations were to be interpreted as: (1) The majority of well-informed individuals would want the suggested course of action, but an appreciable proportion would not; (2) Values and preferences related to this intervention are likely to vary widely;

(3) Policy making will require extensive debates and involvement of many stakeholders.

The panel used the factors listed in Figure 3 as a basis for going from a strong to a weak recommendation. This information was recorded for recommendations where a formal vote was required, as illustrated in Figure 3 .

It took approximately one month to put together a team to prepare the evidence profi les, but once the team was assembled it took only fi ve weeks to prepare and revise the evidence profi les and prepare draft guidelines prior to the panel meeting (Box 1). A draft manuscript for publication was prepared within 10 days following the panel meeting. It took only two additional weeks to complete a fi nal draft of the document (on April 21, 2006 ) that included all of the considered evidence and recommendations.

Overall the quality of the underlying evidence for all recommendations was very low because the evidence was based upon observational data from case series describing small numbers of patients infected with avian infl uenza A (H5N1) virus, laboratory research, or on extrapolation from randomized trials of treatment and prophylaxis for seasonal infl uenza. The panel evaluated the rationale for the quality rating during the meeting and subsequently via electronic correspondence. The reasons for considering the evidence to be of low or very low quality (i.e., for having little confi dence in the available estimates of effect) were: (1) the lack of direct evidence from trials among patients with H5N1 infection or exposure, (2) lack of evidence for important outcomes for H5N1 infection that were not common or not measured in seasonal infl uenza trials, and (3) sparse data for other important outcomes. Additional factors and specifi c judgments regarding the quality of the evidence are included in detailed footnotes in the evidence profi les. The panel expressed concern about the categories used to grade the quality of evidence. For example, although the quality of the evidence for treatment and chemoprophylaxis with oseltamivir and zanamivir was very low for both, there was nonetheless a difference in the quality of the evidence within the categories for these two drugs.

The panel considered several different specifi c patient and exposure groups when considering the balance between benefi ts, harms, and costs for chemoprophylaxis. This led them to develop a risk categorization for exposure to assist decision makers in prioritizing use of antivirals [20, 21] .

Of the 27 recommendations, 15 were strong recommendations, but most (11) were strong recommendations against a specifi c intervention. The arguments for making strong recommendations for specifi c interventions, despite the low quality 

The quality of evidence for each important or critical outcome is based on:

• the study design • limitations of the studies (execution)

• consistency of the evidence across studies

• the directness (generalizability) of the evidence to the population, intervention, comparison, and outcomes

• the precision of the estimate or sparseness of data Evidence was classifi ed as "high", "moderate", "low", or "very low" based on these criteria and the following defi nitions:

• High: Further research is very unlikely to change confi dence in the estimate of effect.

• Moderate: Further research is likely to have an important impact on confi dence in the estimate of effect and may change the estimate.

• Low: Further research is very likely to have an important impact on confi dence in the estimate of effect and is likely to change the estimate.

• Very low: Any estimate of effect is very uncertain.

of the available evidence, were the high risk of serious outcomes including death, the lack of alternative treatments, a low risk of serious adverse effects based on the available evidence, relatively low costs, and a possibility of benefi cial effects, although there is much uncertainty about these. The strong recommendation regarding treatment of H5N1 patients with oseltamivir required a vote (one panel member abstained and one voted for a weak recommendation). The main argument that led to voting about this recommendation was that it is uncertain whether the intervention does more good than harm in the face of very low-quality evidence.

Only one other recommendation required voting. Other factors that infl uenced the recommendations, value judgments, and other directly relevant information are described in the remark sections following each recommendation [21] .

These guidelines were prepared in less than two months through an intense international collaboration, the use of electronic communication, and one panel meeting. However, it took one additional month initially to put together the team that prepared the evidence profi les. There are at least two ways in which the time needed to prepare rapid advice could be shortened: fi rst, by identifying or establishing collaborating centres with the competency needed to prepare evidence profi les and second, by building up in-house capacity to reduce the time needed to organize a review team.

Strengths of the process. Strengths of the applied process include its transparency and the short amount of time used to prepare the guidelines. Guideline developers are increasingly using the GRADE approach because it includes transparent judgments about each of the key factors that determine the quality of evidence for each important outcome, and overall across outcomes for each recommendation [10, 22, 23] . In addition, the approach used in developing these guidelines included transparent consideration of the key factors that determine the strength of a recommendation. The rapid preparation of evidence profi les was possible because of the availability of high-quality systematic reviews of the evidence from seasonal infl uenza and the involvement of a review team with experience in preparing evidence profi les and relevant clinical expertise. In the absence of higher-quality direct evidence, the panel considered case reports, animal studies, and in vitro studies for H5N1, as well as the available indirect evidence from systematic reviews of clinical trials for seasonal infl uenza, systematically and transparently.

The broad representation of stakeholders in the guideline group allowed the inclusion of different perspectives for making informed judgments about the importance of outcomes, the quality of evidence, and the strength of recommendations. The publicly available evidence profi les facilitate adaptation of the guidelines to specifi c settings and updating of the guidelines, as well as contributing to transparency [13, 21, 24] .

Limitations of the process. Limitations of this process relate to its very purpose: providing rapid advice. Thus, the time available for developing these guidelines did not permit detailed consideration of all clinical questions that clinicians may face. For example, the discussions about whether to use prophylactic antibiotics and which recommendations apply to situations of human-to-human transmission were short and, therefore, did not result in specifi c recommendations. It is unlikely that important evidence that would have led to different recommendations was missed given the nature of the problem; i.e., an emerging disease. In areas with an ample evidentiary base, evidence could be missed by relying on systematic reviews of the indirect evidence. However, this generally should not be the case when rapid advice is needed. It is not clear whether the differences the panel identifi ed within the category of very low-quality evidence are truly important for decision making, but panel members requested that this information should not be lost in translating the quality of evidence into the four categories based on the GRADE approach.

The total budget required for this guideline development project amounted to approximately US$150,000, for a focused guideline. This was made possible in part by limiting the number of meetings through the use of electronic communication tools. While this cost compares favourably with the cost of other guideline development processes, it remains out of reach for many countries trying to develop national rapid advice guidelines.

Involvement of stakeholders through consultation is limited by the rapid process. There are at least three ways in which stakeholder involvement could be improved. First, rapid consultations could be facilitated by establishing stakeholder groups and mechanisms for involvement such as those used by the National Centre for Health and Clinical Excellence. Second, evaluation and updating of the guidelines offer opportunities for more stakeholder involvement in revisions of the guidelines. Third, because the guidelines need to be adapted to specifi c settings, stakeholders could be involved in local adaptation processes.

Although research needs were identifi ed by the guideline panel, these do not provide clear guidance for what research should be prioritized to address the most important uncertainties about pharmacological management of H5N1 infection. While this is partly due to the mandate that was given to the panel, recommendations for research should be viewed as an integral part of making recommendations.

In summary, we found that it is feasible to develop evidence-based guidelines systematically and transparently in as little as two months. The cost of doing this is prohibitively high for low-and middle-income countries and it would be wasteful for high-income countries to duplicate this process unnecessarily. WHO, or others developing rapid advice, can therefore provide an important service by using a robust and transparent process that simplifi es adaptation to specifi c settings. Further work is required to develop systematic processes for WHO to give even faster or immediate guidance for emerging infectious diseases.

Text S1. Information on cost and resource utilization, and declaration and handling of confl icts of interest Author contributions. All authors declare that they participated in the development of the methods for rapid advice guidelines and that they have seen and approved the fi nal version of this manuscript. HJS chaired the guideline panel as a voting member, drafted recommendations, contributed to the design and development of the process for this guideline, reviewed the evidence summaries, provided additional evidence, and wrote the fi rst draft of this manuscript. HJS is the guarantor for this article. SRH led the guideline development process, contributed to the design and development of the process for this guideline, reviewed the evidence summaries, provided additional evidence, participated in the guideline panel meeting, critically reviewed early drafts of the manuscript, and approved the fi nal draft of the manuscript. MK prepared and reviewed the evidence summaries, provided additional evidence, participated in the guideline panel meeting, compiled literature, tracked versions of the full guidelines, and critically reviewed and approved the fi nal draft of the manuscript. GEV, LS, and TFW prepared the evidence summaries and critically reviewed and approved the manuscript. RB prepared and reviewed the evidence summaries, provided additional evidence, participated as voting member in the guideline panel meeting, and critically reviewed and approved the fi nal draft of the manuscript. CDM, FH, TMU, JF, YY, JB, TC, TTH, BÖ, and NS reviewed the evidence summaries, provided additional evidence, participated as voting members in the guideline panel meeting, and critically reviewed and approved the fi nal draft of the manuscript. HZ participated in the guideline panel meeting and critically reviewed and approved the fi nal draft of the manuscript. ADO contributed to the design and development of the process of this guideline development, organized and led the evidence review and evidence summary preparation, and critically reviewed and approved the fi nal draft of the manuscript.

Storm over WHO-ISH hypertension guidelines

WHO: The casualties and compromises of renewal

years of the WHO essential medicines lists: Progress and challenges

Critics slam draft WHO report on homoeopathy

Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature

Practice guidelines developed by specialty societies: The need for a critical appraisal

A time to ask what you want of WHO

Attributes of clinical guidelines that infl uence use of guidelines in general practice: Observational study

Practice guidelines: A new reality in medicine. I. Recent developments

Guidelines for WHO guidelines

Users' guide to the medical literature: A manual for evidence-based clinical practice

Methodology for guideline development for the Seventh American College of Chest Physicians Conference on Antithrombotic and Thrombolytic Therapy: The Seventh ACCP Conference on Antithrombotic and Thrombolytic Therapy

Developing clinical guidelines: A challenge to current methods

National Institutes for Health and Clinical Excellence

Evidence-informed health policy: Using research to make health systems healthier. Oslo: Norwegian Knowledge Centre for the Health Services

Research translation, dissemination, and application-Moving toward a new vision and strategic framework. Discussion and consensus fi ndings. NHLBI Cardivascular Thought Leaders Meeting

Independent experts from World Health Organisation to review NICE clinical guidelines programme

Improving the use of research evidence in guideline development: Introduction

AD (2006) Improving the use of research evidence in guideline development: 1. Review of guidelines for guidelines

WHO Rapid Advice Guidelines for the pharmacological management of sporadic human infection with avian infl uenza A (H5N1) virus

WHO Rapid Advice Guidelines on pharmacological management of humans infected with avian infl uenza A (H5N1) virus

Grading quality of evidence and strength of recommendations

An offi cial ATS statement: Grading the quality of evidence and strength of recommendations for guidelines and recommendations

Evidence-based: US road and public-health side of the street

The authors acknowledge the expert administrative support of Julia Brass and Madelyn Clemente. The authors thank Dr. Irene Terrenato and Prof. Paola Muti for