key: cord-0807309-66l8ncgs authors: Tendal, Britta; Vogel, Joshua P.; McDonald, Steve; Norris, Sarah; Cumpston, Miranda; White, Heath; Leder, Karin; Navarro, David Fraile; Cheyne, Saskia; Chakraborty, Samantha; Murano, Melissa; Millard, Tanya; Callesen, Henriette E.; Islam, Rakibul M.; Elliott, Julian; Turner, Tari title: Weekly updates of national living evidence-based guidelines: Methods for the Australian Living Guidelines for Care of People with COVID-19 date: 2020-11-11 journal: J Clin Epidemiol DOI: 10.1016/j.jclinepi.2020.11.005 sha: b54bd3ddfb781116f104cd3c57298c7c6d781f0d doc_id: 807309 cord_uid: 66l8ncgs Background The Australian National COVID-19 Clinical Evidence Taskforce is a consortium of 31 Australian health professional organisations developing living, evidence-based guidelines for care of people with COVID-19 which are updated weekly. This paper describes the methods used to develop and maintain the guidelines. Methods The guidelines use GRADE methods and are designed to meet Australian NHMRC standards. Each week, new evidence is reviewed, current recommendations are revised and new recommendations made. These are published in MAGIC and disseminated through traditional and social media. Relevant new questions to be addressed are continually sought from stakeholders and practitioners. For prioritised questions, the evidence is actively monitored and updated. Evidence surveillance combines horizon scans and targeted searches. An Evidence Team appraises and synthesises evidence and prepares evidence-to-decision frameworks to inform development of recommendations. A Guidelines Leadership Group oversees the development of recommendations by multidisciplinary Guidelines Panels and is advised by a Consumer Panel. Results and Discussion: The Taskforce formed in March 2020, and the first recommendations were published two weeks later. The guidelines have been revised and republished on a weekly basis for 24 weeks, and as of October 2020 contain over 90 treatment recommendations, suggesting that living methods are feasible in this context. Conclusions The Australian Guidelines for Care of People with COVID-19 provide an example of the feasibility of living guidelines, and an opportunity to test and improve living evidence methods. In January 2020, the outbreak of the novel coronavirus SARS-CoV-2 was declared a Public Health Emergency of International Concern. By early April 2020, there were 1.2 million cases of COVID-19 and over 69,000 deaths. (1) In the same period, more than 300 COVID-19 clinical trials were registered on ClinicalTrials.gov, and numbers continue to grow(2). As a novel virus, in the early phases of the pandemic clinicians were required to make vital COVID-19 treatment decisions with limited direct evidence. However, an escalating volume of research related to COVID-19 is released daily,(3) requiring rapid and ongoing synthesis to ensure guidance for health decisions remains up to date. Evidence-based guideline development relies on a series of systematic reviews to identify and appraise the research on which recommendations are based. Recommendations can rapidly become out of date as new research is published but not incorporated into the guidelines, with some recommendations already out of date at the time of publication. (4) (5) (6) Delays in including evidence in guidelines may contribute to sub-optimal quality of care and patient outcomes. 'Living' evidence synthesis methods were developed to enable systematic reviews to be continually updated as new evidence emerges. (7) Living systematic reviews are now conducted across the spectrum of health research (8) , and living approaches are being applied to guideline development (9) , with pilot living guidelines projects underway in stroke, diabetes, maternal health and other areas. (10) (11) (12) A living approach to guideline development includes rapid prioritisation of areas where guidance is needed, continual evidence surveillance and frequent updating of recommendations. By continually incorporating emerging evidence, these methods ensure that currency of guideline recommendations is maintained. (9) Living approaches are appropriate when: 1. Guideline recommendations are a priority for decision-making; and 2. Recommendations are likely to change as new evidence emerges; and 3. New research evidence is likely to become available. (9) The decisions made by clinicians caring for patients with COVID-19 unquestionably fit these three criteria. In late March 2020 the Australian Living Evidence Consortium and Cochrane Australia established the National COVID-19 Clinical Evidence Taskforce. (13) The Taskforce is a partnership of 31 Australian peak health professional bodies (Box 1) whose members provide clinical care to people with COVID-19. Taskforce partners collaborate to develop living, evidence-informed guidelines for primary, hospital and critical care of people with suspected or confirmed COVID-19. The guidelines are updated weekly. In this paper, we describe the methods used to develop and maintain the guidelines. The guidelines provide specific, patient-focused recommendations on management and care of people with suspected or confirmed COVID-19, and chemoprophylaxis for people exposed to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The guideline recommendations are intended to be used by individuals responsible for the management and care of people with COVID-19 in Australia. The guidelines are designed to meet the 2016 Australian National Health and Medical Research Council (NHMRC) standard for clinical practice guidelines (14), and the Taskforce is seeking NHMRC approval. The Taskforce is overseen by a Steering Committee, comprised of representatives from member organisations, which has responsibility for endorsing guideline recommendations. A Guidelines Leadership Group (GLG) oversees daily operations, including the work of guidelines panels, and is advised by a consumer panel. The guideline panels review evidence and make draft recommendations in specific areas of clinical practice, to be approved by the GLG. They are supported by an Evidence Team that identifies, appraises and synthesises evidence (see Figures 1 and 2). Membership of these groups is described on the Taskforce website. (13) 1. Identification and formulation of clinical questions Clinical questions are selected and prioritised for living evidence review where they meet the criteria for living evidence synthesis. Over a two-week period in March 2020, we identified initial questions through: a survey of the membership and discussions with leaders of Taskforce organisations; a review of existing guidelines; and discussions with key stakeholders. Since publication of the first version of the guidelines, questions are sought on an ongoing basis from the guideline and consumer panels, Taskforce members, and through the Taskforce website. Each week, in-scope questions are prioritised by the panels by: 1. Likely impact on patient outcomes 2. Proportion of clinical population impacted 3. Extent of variation in current practice High-priority questions are selected by the GLG. Each priority question is formulated by the Evidence Team using the Patient, Intervention, Comparator, Outcomes (PICO) format, and clinical experts on the panels ensure the questions target areas of urgent clinical uncertainty. Where possible, outcomes are aligned with the COVID-19 Core Outcomes Set. (15) A list of the topics currently addressed by the guidelines is provided in Box 2, and an up-to-date list is available on the guideline website. (13) 2. Search and selection methods An information specialist oversees evidence surveillance and undertakes searches of databases and other sources. To avoid duplication, evidence surveillance combines daily horizon scans of several COVID-19 sources, including organisations producing COVID-19 guidance, plus targeted searches for specific PICO questions as they are prioritised. The purpose of the horizon scan is to be aware of new evidence syntheses (systematic reviews, rapid reviews, living reviews) and primary studies that fall within the scope of the guideline (see Table 1 ). Several auto-alerts are run daily or weekly in PubMed to supplement the horizon-scanning activities. The Evidence Team also liaises with several international groups who are sourcing and conducting research and evidence syntheses within the guideline scope to minimise duplication and ensure rapid access to relevant research. As new questions are prioritised, existing search surveillance methods are checked to ensure they cover the new questions, and the search is updated or expanded as needed. Articles are eligible for inclusion if they report results of primary research or systematic reviews relevant to one of the PICO questions. PICO questions are occasionally structured to include evidence from other, similar diseases (such as other viral pneumonia), to provide additional indirect evidence for formulating recommendations. Specific inclusion and exclusion criteria are developed for each PICO question. Studies retrieved by the search are uploaded into Covidence for screening by at least two members of the Evidence Team. Disagreements are resolved through discussion and, if necessary, a third reviewer adjudicates. Full-text articles are screened using the same process, with expert clinical input as required. The guideline uses GRADE (Grading of Recommendations Assessment, Development and Evaluation) (16, 17) methodology, as it is recommended as best practice by the NHMRC, the body responsible for approving clinical practice guidelines in Australia, and is used by many international organisations, including the World Health Organization and Cochrane, enabling collaboration.(18, 19) GRADE is a transparent framework for developing and presenting summaries of evidence and provides a systematic approach for making clinical practice recommendations. Development is supported by the online guideline development and publication platform 'MAGIC' (Making GRADE the Irresistible Choice).(16, 17) Using MAGIC allows collaboration across locations, and rapidly publish updated recommendations in an easily accessible web format. The Evidence Team extracts data using standardised forms. Data are extracted independently by two reviewers and disagreements are resolved through discussion and, if necessary, a third reviewer adjudicates. Extracted data are cross-referenced with an outcome matrix to ensure that all important outcomes are analysed, and we cross-check our assessments with other groups undertaking rigorous syntheses. A summary table of study characteristics is used to group studies for analysis, in accordance with pre-specified questions. Systematic reviews are assessed using AMSTAR2 (20) , and the risk of bias assessment of included studies from the review is used where available. Individual primary studies are assessed using the Cochrane Risk of Bias 2.0 tool for randomised trials (21) and the ROBINS-I tool for non-randomised studies. (22) Where an existing systematic review is available and up to date, additional synthesis is not undertaken, and reviews are monitored for further updates. Where an existing systematic review is available but not up to date (or becomes out of date), a limited update may be conducted to integrate new studies. Where primary studies are available to answer a clinical question, a systematic review is initiated and synthesis methods selected as appropriate to the question, in accordance with Cochrane standards. (21) For each question, the Evidence Team develops evidence-to-decision tables following GRADE processes. Where needed, specialist expertise is sought from the Expert Advisory Group. The complete evidence profiles and rationales for recommendations are included in the guidelines. Seven guideline panels have been convened: • Primary and chronic care • Hospital and acute care • Critical care • Pregnancy and perinatal care • Paediatric and adolescent care • Palliative and aged care panel • Disease-modifying treatment and chemoprophylaxis Panels are intentionally diverse across disciplines, setting, geography and gender. Members of panels are sought through consultation with Taskforce members and calls for expressions of interest. New panel members are recruited, and new panels convened, as areas of need arise. Panels discuss and revise draft recommendations within their scope at weekly meetings. Panels consider each of the GRADE domains, including benefits and harms, the certainty of the evidence, resources, feasibility, acceptability and equity in formulating their decisions. Consumer representatives particularly consider whether strong or varying patient preferences and values are likely to impact on the nature or implementability of the recommendations. A recommendation is rated: • strong when most or all individuals will be best served by the recommended course of action • conditional when not all individuals will be best served by the recommended course of action and there is a need to consider the individual patient's circumstances, preferences, and values. For some topics, there is insufficient evidence on which to base a recommendation, however panels believe it important to provide advice. Advice about these topics is developed based on consensus expert opinion (guided by any relevant evidence) and labelled as Consensus Recommendations or Practice Statements. Draft recommendations from the panels are reviewed and approved by the GLG and endorsed by the Steering Committee before publication. On publication, each recommendation is labelled as 'Updated' or 'New'. Updated recommendations are existing recommendations where the strength or the direction of the recommendation has changed. Care is taken to coordinate the work of the panels and ensure consistency in recommendations across panels. Guideline panel chairs and Evidence Team members meet regularly, and panel chairs sit on the GLG where they review the recommendations across each of the panels. The Taskforce has partnered with the Consumers Health Forum of Australia, the peak national health consumer body, to co-convene a consumer panel. The panel consists of eight to ten consumers who advise the GLG on: • new clinical questions and high-priority topics, • relative importance of different outcomes, and • feedback on guideline recommendations. The co-chairs of the Consumer Panel also represent the panel on the GLG. The Taskforce is mindful that Indigenous peoples or other population groups (including culturally and linguistically diverse communities) may have specific needs in relation to COVID-19 care. Our guideline panels include a diversity of representation, including Indigenous peoples and remote and regional health practitioners. The National Aboriginal Community Controlled Health Organisation and the Australian College of Rural and Remote Medicine are both members of the Taskforce. Searches cover all population groups, with no limits on the patients or population of studies, other than that they must have a diagnosis or suspicion of COVID-19. In the early phase of the Taskforce, it became evident that specialty panels would be required for specific sub-populations. The Taskforce established separate panels for pregnancy and perinatal care, paediatric and adolescent care, and palliative and aged care. These panels develop recommendations specific to these population (for example, care during labour and childbirth), and adapt existing recommendations (for example, use of antiviral medicines in infants). Weekly guideline updates are disseminated through general and health media, including traditional and social media channels, and distributed to the members of the Taskforce for dissemination. Each week, a 'Taskforce Communiqué' is sent to all member organisations, partners and funders of the Taskforce, as well as subscribers to the Taskforce website (and is available online on the Taskforce website (13)). The Communiqué provides an overview of the work of the Taskforce for that week and includes content that Taskforce members can adapt for communications with their members. A process evaluation is being undertaken to continually improve processes and outputs of the Taskforce and guidelines project, and inform future living guideline projects. Each month, experiences of participants in the Taskforce are captured through online surveys, semi-structured interviews and a stocktake of activity. Results are provided monthly to the Taskforce Steering Committee. An impact evaluation will examine the influence of the guidelines on care of people with COVID-19, identifying the extent to which health decision-makers were aware of the guidelines; valued the guidelines; and used the guidelines to inform decision-making. The Taskforce was formed in late March 2020, the first version of the guidelines was published two weeks later and included 10 recommendations. Using the methods described above, the guidelines have been revised, updated and republished each week since. By early October, the evidence team of 11 full-time equivalent staff, working with seven guideline panels and over 230 individuals (clinicians, policymakers, and others) had screened more than 5000 citations, appraised and synthesised 53 randomised trials, 13 non-randomised studies and 12 systematic reviews. Collectively this evidence had informed 92 recommendations addressing 81 topics (version 24, 1 October 2020). The Australian Guidelines for the Management and Care of People with COVID-19 provide a largescale test case for living approaches to GRADE-based guideline development. The weekly update schedule is the most frequent of which we are aware, and significantly more rapid than other living guidelines projects. (10) (11) (12) This is commensurate with the extraordinarily rapid emergence of evidence related to management of COVID-19.(23) Our experience of 24 successful weekly cycles suggests that very frequent updating is feasible, even during a time of significant disruption. The short update cycles require all contributors to be agile and act collaboratively, and depend on strong project management. Others have noted that rapid reviews are particularly challenging during COVID-19 (24) , and the team has faced challenges in developing the guidelines. As a new Taskforce, the project team moved quickly to both rapidly support recommendation development and publication, while simultaneously establishing infrastructure, recruiting staff, and developing the workflows to deliver living guidelines. It took several weeks to establish stability in the team and processes. Having a team of experienced methodologists familiar with living evidence synthesis methods was crucial. With all team members J o u r n a l P r e -p r o o f working remotely, technologies such as Zoom and Slack have been indispensable in enabling the rapid workflows. Globally, several groups have also recognised the particular value of living evidence synthesis approaches in the context of a novel disease with such a swiftly evolving evidence base. Cochrane has rapidly produced living systematic reviews (LSRs) addressing the value of convalescent plasma and rehabilitation interventions for people with COVID-19 (25, 26) , on signs and symptoms, and antibody tests to identify people with COVID-19 (27, 28) . The Cochrane convalescent plasma LSR provides a clear demonstration of why these approaches are useful. The initial version, published in May, included only eight studies, predominantly case series, with a total of just 32 participants. The first update, published two months later, included 20 studies (one randomised and 19 nonrandomised) with over 5000 participants, and also identified 50 ongoing randomised trials. (25) Groups The existence of LSRs produced by international research groups and other pre-filtered and preappraised evidence sources, including COVID-19-specific research databases, was a major enabler of our work and reduced duplication of effort. Living methods are also beginning to be applied to other guideline development for COVID-19, for example the World Health Organization guideline for COVID-19 drugs (37) and the BMJ Rapid Recommendation on remdesivir (38) , however we are not aware of other living guideline programs which match the scope or frequency of updating achieved by the Taskforce. The Taskforce brings together 31 Australian peak health professional bodies whose members provide care to people with COVID-19. The willingness of these organisations to collaborate, quickly identify representatives and panel members, and establish rapid processes for recommendation endorsement has been crucial to the successful delivery of weekly guideline updates. Similarly, the commitment of individual clinicians and others to weekly panel meetings has been outstanding, especially given increased workloads resulting from the pandemic. Support from NHMRC was also vital in ensuring the guidelines meet NHMRC standards. The Australian Guidelines for the Management and Care of People with COVID-19 provide an important example of the feasibility of rapid living GRADE-based guideline development, and an opportunity to robustly test and improve living guideline development methods. Results of the process and impact evaluations will provide useful insights to guide future work in this area. As of September, the guideline recommendations cover: Living evidence map and living systematic review of Covid-19 studies (covid-nma.org) Identifies randomised trials, non-randomised studies and case series from daily screening of searches of PubMed, Chinarxiv and MedRxiv. Provides study characteristics, risk of bias assessments and forest plots. Other sources COVID-19: What to know about the coronavirus pandemic on 6 April: World Economic Forum Characteristics of registered clinical trials assessing treatments for COVID-19: a cross-sectional analysis Research methodology and characteristics of journal articles with original data, preprint articles and registered clinical trial protocols about COVID-19 Median life span of a cohort of National Institute for Health and Care Excellence clinical guidelines was about 60 months The validity of recommendations from clinical guidelines: a survival analysis Durability of class I American College of Heart Association clinical practice guideline recommendations Living systematic review: 1. Introduction-the why, what, when, and how Methods of conduct and reporting of living systematic reviews: a protocol for a living methodological survey Living Systematic Review N. Living systematic reviews: 4. Living guideline recommendations Bringing stroke clinical guidelines to life Breathing life into Australian diabetes clinical guidelines Developing and applying a 'living guidelines' approach to WHO recommendations on maternal and perinatal health NHMRC Standards for Guidelines: National Health and Medical Research Council (NHMRC) Core outcomes set for people with COVID-19 GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction World Health Organization. WHO Handbook for Guideline Development (second edition). Geneva: World Health Organization AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both Cochrane Handbook for Systematic Reviews of Interventions version 6 RoB 2: a revised tool for assessing risk of bias in randomised trials Keep up with the latest coronavirus research Rapid review methods more challenging during COVID-19: Commentary with a focus on 8 knowledge synthesis steps Convalescent plasma or hyperimmune immunoglobulin for people with COVID-19: a living systematic review Rehabilitation and COVID-19: the Cochrane Rehabilitation 2020 rapid living systematic review. Update as of Signs and symptoms to determine if a patient presenting in primary care or hospital outpatient settings has COVID-19 disease Update to living systematic review on prediction models for diagnosis and prognosis of covid-19 Systematic rapid living review on rehabilitation needs due to COVID-19: update to May 31st, 2020 Vitamin C for COVID-19: A living systematic review Systematic rapid living review on rehabilitation needs due to COVID-19: update as of April 30th, 2020 Hydroxychloroquine or Chloroquine for Treatment or Prophylaxis of COVID-19: A Living Systematic Review Ventilation Techniques and Risk for Transmission of Coronavirus Disease, Including COVID-19: A Living Systematic Review of Multiple Streams of Evidence Verdugo-Paiva F, Izcovich A, Ragusa M, Rada G. Lopinavir-ritonavir for COVID-19: A living systematic review A living WHO guideline on drugs for covid-19 Remdesivir for severe covid-19: a clinical practice guideline All authors developed and documented the methods described in the article on behalf of the National COVID-19 Clinical Evidence Taskforce. TT prepared the first draft of the manuscript and incorporated the feedback to produce the submitted version of the manuscript, which all of the authors approved.