key: cord-0947618-al5rw1kq authors: Yao, Liang; Ahmed, Muhammad Muneeb; Guyatt, Gordon H; Yan, Peijing; Hui, Xu; Wang, Qi; Yang, Kehu; Tian, Jinhui; Djulbegovic, Benjamin title: Discordant and inappropriate discordant recommendations in consensus and evidence based guidelines: empirical analysis date: 2021-11-25 journal: BMJ DOI: 10.1136/bmj-2021-066045 sha: e6e0c8e1d4458021ee05858d91fa2c63567850ca doc_id: 947618 cord_uid: al5rw1kq OBJECTIVE: To investigate whether alignment of strength of recommendations with quality of evidence differs in consensus based versus evidence based guidelines. DESIGN: Empirical analysis. DATA SOURCE: Guidelines developed by the American College of Cardiology and the American Heart Association (ACC/AHA) and the American Society of Clinical Oncology (ASCO) up to 27 March 2021. STUDY SELECTION: Recommendations were clearly categorised as consensus or evidence based, were separated from the remainder of the text, and included both the quality of evidence and the strength of the recommendations. DATA EXTRACTION: Paired authors independently extracted the recommendation characteristics, including type of recommendation (consensus or evidence based), grading system used for developing recommendations, strength of the recommendation, and quality of evidence. The study team also calculated the number of discordant recommendations (strong recommendations with low quality evidence) and inappropriate discordant recommendations (those that did not meet grading of recommendations assessment, development, and evaluation criteria of appropriateness). RESULTS: The study included 12 ACC/AHA guidelines that generated 1434 recommendations and 69 ASCO guidelines that generated 1094 recommendations. Of the 504 ACC/AHA recommendations based on low quality evidence, 200 (40%) proved to be consensus based versus 304 (60%) evidence based; of the 404 ASCO recommendations based on low quality evidence, 292 (72%) were consensus based versus 112 (28%) that were evidence based. In both ACC/AHA and ASCO guidelines, the consensus approach yielded more discordant recommendations (ACC/AHA: odds ratio 2.1, 95% confidence interval 1.5 to 3.1; ASCO: 2.9, 1.1 to 7.8) and inappropriate discordant recommendations (ACC/AHA: 2.6, 1.7 to 3.7; ASCO: 5.1, 1.6 to 16.0) than the evidence based approach. CONCLUSION: Consensus based guidelines produce more recommendations violating the evidence based medicine principles than evidence based guidelines. Ensuring appropriate alignment of quality of evidence with the strength of recommendations is key to the development of “trustworthy” guidelines. Evidence-based There was sufficient evidence from published studies to inform a recommendation to guide clinical practice Formal Consensus The available evidence was deemed insufficient to inform a recommendation to guide clinical practice. Therefore, the expert Panel used a formal consensus process to reach this recommendation, which is considered the best current guidance for practice. The Panel may choose to provide a rating for the strength of the recommendation (i.e., "strong," "moderate," or "weak"). The results of the formal consensus process are summarized in the guideline and reported in an online data supplement. Informal Consensus The available evidence was deemed insufficient to inform a recommendation to guide clinical practice. The recommendation is considered the best current guidance for practice, based on informal consensus of the expert Panel. The Panel agreed that a formal consensus process was not necessary for reasons described in the literature review and discussion. The Panel may choose to provide a rating for the strength of the recommendation (i.e., "strong," "moderate," or "weak"). High High confidence that the available evidence reflects the true magnitude and direction of the net effect (e.g., balance of benefits versus harms) and further research is very unlikely to change either the magnitude or direction of this net effect. Intermediate confidence that the available evidence reflects the true magnitude and direction of the net effect. Further research is unlikely to alter the direction of the net effect, however it might alter the magnitude of the net effect. Low confidence that the available evidence reflects the true magnitude and direction of the net effect. Further research may change the magnitude and/or direction of this net effect. Evidence is insufficient to discern the true magnitude and direction of the net effect. Further research may better inform the topic. Reliance on consensus opinion of experts may be reasonable to provide guidance on the topic until better evidence is available. There is high confidence that the recommendation reflects best practice. This is based on: a) strong evidence for a true net effect (e.g., benefits exceed harms); b) consistent results, with no or minor exceptions; c) minor or no concerns about study quality; and/or d) the extent of panelists' agreement. Other compelling considerations (discussed in the guideline's literature review and analyses) may also warrant a strong recommendation There is moderate confidence that the recommendation reflects best practice. This is based on: a) good evidence for a true net effect (e.g., benefits exceed harms); b) consistent results, with minor and/or few exceptions; c) minor and/or few concerns about study quality; and/or d) the extent of panelists' agreement. Other compelling considerations (discussed in the guideline's literature review and analyses) may also warrant a moderate recommendation. There is some confidence that the recommendation offers the best current guidance for practice. This is based on: a) limited evidence for a true net effect (e.g., benefits exceed harms); b) consistent results, but with important exceptions; c) concerns about study quality; and/or d) the extent of panelists' agreement. Other considerations (discussed in the guideline's literature review and analyses) may also warrant a weak recommendation. Initial Management of Noncastrate Advanced, Recurrent, or Metastatic Prostate Cancer: ASCO Guideline Update Management of Dyspnea in Advanced Cancer: ASCO Guideline Neoadjuvant Chemotherapy, Endocrine Therapy, and Targeted Therapy for Breast Cancer: ASCO Guideline Therapy for Stage IV Non-Small-Cell Lung Cancer With Driver Alterations: ASCO and OH (CCO) Joint Guideline Update Diagnosis and Management of Squamous Cell Carcinoma of Unknown Primary in the Head and Neck: ASCO Guideline Estrogen and Progesterone Receptor Testing in Breast Cancer: ASCO/CAP Guideline Update Germline and Somatic Tumor Testing in Epithelial Ovarian Cancer: ASCO Guideline Lung Cancer Surveillance After Definitive Curative-Intent Therapy: ASCO Guideline Management of Cancer Cachexia: ASCO Guideline American Society for Radiation Oncology, and Society of Surgical Oncology Guideline Management of Male Breast Cancer: ASCO Guideline Molecular Biomarkers in Localized Prostate Cancer: ASCO Guideline Optimum Imaging Strategies for Advanced Prostate Cancer: ASCO Guideline Prevention and Management of Chemotherapy-Induced Peripheral Neuropathy in Survivors of Adult Cancers: ASCO Guideline Update Systemic Therapy for Melanoma: ASCO Guideline Therapy for Stage IV Non-Small-Cell Lung Cancer Without Driver Alterations: ASCO and OH (CCO) Joint Guideline Update Adjuvant Therapy for Resected Biliary Tract Cancer: ASCO Clinical Practice Guideline Duration of Oxaliplatin-Containing Adjuvant Therapy for Stage III Colon Cancer: ASCO Clinical Practice Guideline Management of Cancer-Associated Anemia With Erythropoiesis-Stimulating Agents: ASCO/ASH Clinical Practice Guideline Update Management of Osteoporosis in Survivors of Adult Cancers With Nonmetastatic Disease: ASCO Clinical Practice Guideline Management of the Neck in Squamous Cell Carcinoma of the Oral Cavity and Oropharynx: ASCO Clinical Practice Guideline Medication-Related Osteonecrosis of the Jaw: MASCC/ISOO/ASCO Clinical Practice Guideline Potentially Curable Pancreatic Adenocarcinoma: ASCO Clinical Practice Guideline Update Role of Patient and Disease Factors in Adjuvant Systemic Therapy Decision Making for Early-Stage, Operable Breast Cancer: Update of the ASCO Endorsement of the Cancer Care Ontario Guideline Treatment of Multiple Myeloma: ASCO and CCO Joint Clinical Practice Guideline Use of Biomarkers to Guide Decisions on Adjuvant Systemic Therapy for Women With Early-Stage Invasive Breast Cancer: ASCO Clinical Practice Guideline Update-Integration of Results From TAILORx Use of Endocrine Therapy for Breast Cancer Risk Reduction: ASCO Clinical Practice Guideline Update Venous Thromboembolism Prophylaxis and Treatment in Patients With Cancer: ASCO Clinical Practice Guideline Update Antimicrobial Prophylaxis for Adult Patients With Cancer-Related Immunosuppression: ASCO and IDSA Clinical Practice Guideline Update Summary Evaluating Susceptibility to Pancreatic Cancer: ASCO Provisional Clinical Opinion Hypofractionated Radiation Therapy for Localized Prostate Cancer: An ASTRO, ASCO, and AUA Evidence-Based Guideline Management of Immune-Related Adverse Events in Patients Treated With Immune Checkpoint Inhibitor Therapy Metastatic Pancreatic Cancer: ASCO Clinical Practice Guideline Update Optimizing Anticancer Therapy in Metastatic Non-Castrate Prostate Cancer Outpatient Management of Fever and Neutropenia in Adults Treated for Malignancy: American Society of Clinical Oncology and Infectious Diseases Society of America Clinical Practice Guideline Update Palliative Care in the Global Setting: ASCO Resource-Stratified Practice Guideline Practical Assessment and Management of Vulnerabilities in Older Patients Receiving Chemotherapy: ASCO Guideline for Geriatric Oncology Summary Recommendations on Disease Management for Patients With Advanced Human Epidermal Growth Factor Receptor 2-Positive Breast Cancer and Brain Metastases: ASCO Clinical Practice Guideline Update Selection of Optimal Adjuvant Chemotherapy and Targeted Therapy for Early Breast Cancer: ASCO Clinical Practice Guideline Focused Update Sentinel Lymph Node Biopsy and Management of Regional Lymph Nodes in Melanoma Systemic Therapy for Patients With Advanced Human Epidermal Growth Factor Receptor 2-Positive Breast Cancer: ASCO Clinical Practice Guideline Update Treatment of Malignant Pleural Mesothelioma Use of Larynx-Preservation Strategies in the Treatment of Laryngeal Cancer: American Society of Clinical Oncology Clinical Practice Guideline Update Adjuvant Systemic Therapy and Adjuvant Radiation Therapy for Stage I to IIIA Completely Resected Non-Small-Cell Lung Cancers: American Society of Clinical Oncology/Cancer Care Ontario Clinical Practice Guideline Update of Clinical Oncology Clinical Practice Guideline Update Guideline for the Management of Fever and Neutropenia in Children With Cancer and Hematopoietic Stem-Cell Transplantation Recipients: 2017 Update Management of Small Renal Masses Molecular Biomarkers for the Evaluation of Colorectal Cancer : Guideline From the Patient-Clinician Communication Platelet Transfusion for Patients With Cancer: American Society of Clinical Oncology Clinical Practice Guideline Update Summary Prevention and Monitoring of Cardiac Dysfunction in Survivors of Adult Cancers Primary Prevention of Cervical Cancer Radiation Therapy for Oropharyngeal Squamous Cell Carcinoma: American Society of Clinical Oncology Endorsement of the American Society for Radiation Oncology Evidence-Based Clinical Practice Guideline Role of Bone-Modifying Agents in Metastatic Breast Cancer: An American Society of Clinical Oncology-Cancer Care Ontario Focused Guideline Update Systemic Therapy for Stage IV Non-Small-Cell Lung Cancer: American Society of Clinical Oncology Clinical Practice Guideline Update Treatment of Non-Metastatic Muscle-Invasive Bladder Cancer: AUA/ASCO/ASTRO/SUO Guideline Endocrine Therapy for Hormone Receptor-Positive Metastatic Breast Cancer HER2 Testing and Clinical Decision Making in Gastroesophageal Adenocarcinoma: Guideline From the College of American Pathologists Integration of Palliative Care Into Standard Oncology Care: American Society of Clinical Oncology Clinical Practice Guideline Update Unresectable Pancreatic Cancer Management of Chronic Pain in Survivors of Adult Cancers Advanced Ovarian Cancer: Society of Gynecologic Oncology and American Society of Clinical Oncology Clinical Practice Guideline American Society for Radiation Oncology, and Society of Surgical Oncology Focused Guideline Update Sentinel Lymph Node Biopsy for Patients With Early-Stage Breast Cancer: American Society of Clinical Oncology Clinical Practice Guideline Update Recommendations for the Use of WBC Growth Factors: American Society of Clinical Oncology Clinical Practice Guideline Update Use of Biomarkers to Guide Decisions on Systemic Therapy for Women With Metastatic Breast Cancer Chemotherapy and Targeted Therapy for Women With Human Epidermal Growth Factor Receptor 2-Negative (or unknown) Advanced Breast Cancer Systemic Therapy in Men With Metastatic Castration-Resistant Prostate Cancer: American Society of Clinical Oncology and Cancer Care Ontario Clinical Practice Guideline Screening for Prostate Cancer With Prostate-Specific Antigen Testing 2020 ACC/AHA Guideline for the Management of Patients With Valvular Heart Disease: A Report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines AHA/ACC Guideline for the Diagnosis and Treatment of Patients With Hypertrophic Cardiomyopathy: A Report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines ACC/AHA/HRS Guideline on the Evaluation and Management of Patients With Bradycardia and Cardiac Conduction Delay: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines and the Heart Rhythm Society AHA/ACC Guideline for the Manageme nt of Adults With Congenital Heart Disease: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines Guideline on the Management of Blood Cholesterol: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines ACC/AHA Guideline on the Primary Prevention of Cardiovascular Disease: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines AHA/ACC/HRS Focused Update of the 2014 AHA/ACC/HRS Guideline for the Management of Patients With Atrial Fibrillation: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines and the Heart Rhythm Society Guideline for the Prevention, Detection, Evaluation, and Management of High Blood Pressure in Adults: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines AHA/ACC/HRS Guideline for Management of Patients With Ventricular Arrhythmias and the Prevention of Sudden Cardiac Death: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines and the Heart Rhythm Society AHA/ACC Guideline on the Management of Patients With Lower Extremity Peripheral Artery Disease: Executive Summary: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines ACC/AHA/HRS Guideline for the Evaluation and Management of Patients With Syncope: Executive Summary: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines and the Heart Rhythm Society ACCF/AHA/HRS focused update incorporated into the ACCF/AHA/HRS 2008 guidelines for device-based therapy of cardiac rhythm abnormalities: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines and the Heart Rhythm Society