key: cord-1036503-vkwxj20p authors: Haddaway, Neal R.; Akl, Elie A.; Page, Matthew J.; Welch, Vivian A.; Keenan, Ciara; Lotfi, Tamara title: Open Synthesis and the coronavirus (COVID-19) pandemic in 2020 date: 2020-07-01 journal: J Clin Epidemiol DOI: 10.1016/j.jclinepi.2020.06.032 sha: bdb0588e96f9f273e73a6669312fc2036c0a6524 doc_id: 1036503 cord_uid: vkwxj20p • Open Science principles are vital for ensuring reproducibility, trust and legacy; • Evidence synthesis is a vital means of summarising research for decision-making; • Open Synthesis is the application of Open Science principles to evidence synthesis; • Open approaches to planning, conducting and reporting synthesis have many benefits; • We call on the evidence synthesis community to embrace Open Synthesis. The coronavirus disease 2019 (COVID-19) pandemic of 2020 has caused high levels 30 of mortality and continues to threaten the lives of the global population (1) . The 31 pandemic has amounted to a 'once in a lifetime' event for humanity, and has 32 affected it across its different sectors of existence: health, education, economy, 33 environement, etc. The pandemic continues to threaten job prospects for millions of 34 people, and has resulted in widespread economic turmoil (2). It has also led to the 35 cancellation of numerous conferences (e.g. 3) and research fieldwork, and closed 36 offices across the globe. 37 As the scientific community grapples to respond to the massive and rapidly 39 evolving crisis, the volume of research literature that has been published in relation 40 to the outbreak has expanded rapidly ( Figure 1 ). Simultaneously, efforts to 41 synthesise this growing evidence base have begun, both through ongoing traditional 42 approaches to independent systematic reviews (e.g. 4,5), and through both rapid and 43 living systematic reviews (e.g. Week of 2020 Record count As the volume of evidence increases and decision-makers and scientists struggle to 59 grapple with the rapidly expanding evidence base, many research groups are 60 volunteering to support these efforts by using online collaborative tools and virtual 61 workspaces, in an effort to support continued working during challenging times, 62 and also to help identify, map, and synthesise research as it emerges. 63 This work faces a suite of challenges because of the often closed nature of science. 65 The major challenges are the duplication of efforts (leading to research waste), the 66 inefficiency in conduting research, and missing the opportunity to address 67 important questions. Open Science principles present an opportunity to address these 68 challenges in the context of the COVID-19 pandemic. They would also ensure that 69 the research in the field is more collaborative, transparent, and rigourous. This paper 70 argues for, and illustrates how, to apply the principles of Open Science to the field of 71 evidence synthesis, a concept we refer to as Open Synthesis (6). We use the COVID-72 19 pandemic as a case-in-point to highlight the potential significant benefits of 73 Openness to the research, policy and practice communities. 74 75 Evidence synthesis is the name for research methodologies that involve identifying, 77 collating, appraising, and summarising a body of research evidence using tried-and-78 tested systematic and robust literature review methods: i.e. systematic reviews and 79 systematic maps (7). Systematic reviews are now widely used in the field of 80 healthcare as a 'gold standard' for summarising evidence to provide support for 81 decision-making in policy and practice, through a variety of knowledge translation 82 producuts and practice guidelines (8). 83 However, systematic reviewers face challenges as a result of an often closed 85 academic system: research can be difficult to find and download without access to 86 expensive bibliographic databases (9); primary research articles and the systematic 87 reviews that synthesise them are hidden behind paywalls (10,11); reporting of 88 methods used in trials and syntheses is often deficient to some degree, hampering 89 verification and learning about methodlogy (12); research data is often not made 90 public, particularly when produced by organisations with commercial interests, such 91 as pharmaceutical companies (13); analytical code is rarely shared and statistical 92 methods can be hard to verify (14); and educational materials to train the next 93 generation of evidence synthesists are often not made public (15). 94 Open Science has central premises relating to accessibility and the collaborative 97 nature of knowledge creation and the knowledge itself (16). These principles (see 98 Table 1 ) include concepts such as Open Access (unrestricted availability of research 99 publications; ,11) and Open Data (freely accessible research data used in analyses; 100 ,17) that together support efficient, transparent, and rigorous research. 101 There are various definitions of Open Science, ranging from relatively simple 103 classifications of "data, analysis, publications, and comments" (18), to somewhat 104 more elaborate frameworks (see Table 1 ), all the way to complex hierarchical 105 conceptual models (19). Although these classifications differ in their complexity, 106 they each attempt to cover all aspects of research processes from initiation to 107 communication. 108 109 publication fees, the use of free data repositories) and by incentivising and institutionalising Open and transparent practices from an early career stage (e.g. good code annotation practices). However, this point is not trivial and highlights the need for careful planning across all aspects of Open Synthesis: planning can significantly reduce resource requirements. Standardising methods, processes and tools used to abstract and store data could assist in this process (33) Risk of 'platform capitalism' (i.e. commercialisation of public data) (34) The free availability of data permits the development of subscription-based/ pay-touse services (e.g. Academia.edu) that aim to provide additional services using public data (e.g. analytics) and platforms that may exploit or disadvantage certain groups of people (e.g. by charging for a service that is otherwise already free elsewhere) Grass roots, no-cost alternatives to these services are often available, but awareness of free-to-use services is vital to avoid entrapment by commercial enterprises (e.g. paying a publisher to access an article that is already Open Access). In addition, there are risks associated with some of the practices that may be 234 facilitated by Open Synthesis: for example, 1) living systematic reviews may involve 235 repeated incremental re-running of meta-analyses, leading to increased chances of 236 false positive that need to be accounted for (e.g. 44); 2) updates may need to account 237 for changes in best practice in risk of bias assessments as novel methods become 238 available, potentially involving reassessment of studies identified in the original 239 review. 240 These are not problems with Open Synthesis, but rather important issues that should 242 be addressed when planning incentives and infrastructure in support of Open 243 Syntheses. However, a pathway to Open systematic reviews and systematic maps 244 will involve many steps and a diverse array of different actions: these changes 245 should not be expected overnight and there is a need for detailed discussion about 246 implications and pitfalls. That said, it is generally accepted that the advantages of 247 Open Science outweigh the disadvantages (46). 248 249 At present, some of these Open Synthesis practices are enforced or encouraged by 251 review coordinating bodies. Cochrane reviews can be made immediately Open 252 Access at the point of publication for a fee (payable by authors), or made free after a 253 12 month period (otherwise requiring subscription to access; green Open Access). 254 Cochrane does not yet require systematic review extracted data to be made public 255 We call for increasing application of Open Science and Open Synthesis principles 296 across disciplines both within and beyond the COVID-19 epidemic to support 297 evidence production, synthesis and evidence-informed policy. By embracing Open 298 Synthesis, evidence synthesis communities from all disciplines can maximise the 299 efficiency, impact and legacy of systematic reviews and better support decision- World Health Organization. Coronavirus disease 2019 (COVID-19): situation 319 report The Global Macroeconomic Impacts of COVID-19: 321 Seven Scenarios STAT's guide to health care conferences disrupted by the 324 coronavirus crisis Novel Coronavirus (COVID-19) Outbreak: A Review of the Current 329 Literature COVID-19): A Systematic Review of Imaging Findings in 919 Patients Open Synthesis: on the need for evidence synthesis to embrace 335 Open Science An introduction to systematic reviews GRADE Evidence to Decision (EtD) frameworks: a systematic 339 and transparent approach to making well informed healthcare choices. 1: 340 Introduction 342 Systematic searching for environmental evidence using multiple tools and 343 sources Microfracture produces inferior 345 outcomes to other cartilage repair techniques in chondral injuries in the 346 paediatric knee The 348 state of OA: a large-scale analysis of the prevalence and impact of Open Access 349 articles 351 Reducing waste from incomplete or unusable reports of biomedical research 354 Pathways to independence: towards producing and using trustworthy 355 evidence From the "Replicability Crisis" to Open 357 Science Practices Open education and critical pedagogy. Learning, Media and 362 Technology Open Science: One Term, Five Schools of Thought Opening Science: The Evolving Guide on How the 365 Internet is Changing Research, Collaboration and Scholarly Publishing 366 Data sharing: An open mind on open data Open Science Framework (OSF) Open Science Taxonomy. figshare. Figure Evidence Aid -from the Asian tsunami to the Wenchuan earthquake Using 377 GRADE in situations of emergencies and urgencies: Certainty in evidence and 378 recommendations matters during the COVID-19 pandemic, now more than ever 379 and no matter what The Systematic Review 383 Data Repository (SRDR): descriptive characteristics of publicly available data 384 and opportunities for research. Syst Rev EviAtlas: a tool for visualising evidence synthesis 387 databases. Environmental Evidence Reporting of Financial and Non-financial Conflicts of Systematic Reviews on Health Policy and Systems Research: A Cross Sectional 391 Survey Open Science: A New "Trust 393 Technology"? Science Communication Reinventing discovery: the new era of networked science Mapping the hinterland: 397 Data issues in open science Open science challenges, benefits and tips in early career 399 and beyond Data Sharing Open Science and its Discontents | Ronin Institute Fortifying the Corrective Nature 406 of Post-publication Peer Review: Identifying Weaknesses, Use of Journal Clubs, 407 and Rewarding Conscientious Behavior Keeping research data safe Evidence synthesis 2.0: when 414 systematic, scoping, rapid, living, and overviews of reviews come together Open science: human emancipation or bureaucratic serfdom? 419 SCIRES-it Creative Commons Licences, the Copyright Regime and the Online 422 Community: Is there a Fatal Disconnect? The Modern Law Review Impact of Open Data Policies on 425 Consent to Participate in Human Subjects Research: Discrepancies between 426 Enabling Open-429 Science Initiatives in Clinical Psychology and Psychiatry Without Sacrificing 430 Patients' Privacy: Current Practices and Future Challenges Sharing data-taming 433 the beast: barriers to meta-analyses of individual patient data (IPD) and 434 solutions Institutional inertia and barriers to the adoption of 437 open science Badges to Acknowledge Open Practices: A Simple Effective Method for Increasing Transparency Open 445 science isn't always open to all scientists Multiplicity in randomised trials I: endpoints and 449 treatments. The Lancet Living Systematic Reviews: towards real-time 451 evidence for health-care decision-making | BMJ Best Practice 456 ROBIS: A new tool to assess risk of bias in systematic reviews was developed Benefits of open and high-powered research 459 outweigh costs Why 461 Cochrane should prioritise sharing data Methodological Expectations of Cochrane Intervention Reviews (MECIR) 464 project Campbell systematic reviews takes next step to meeting FAIR 468 principles Highlights • Open Science principles are vital for ensuring reproducibility, trust and legacy • Evidence synthesis is a vital means of summarising research for decision-making • Open Synthesis is the application of Open Science principles to evidence synthesis • Open approaches to planning, conducting and reporting synthesis have many Formal analysis, Funding acquisition, Methodology, Visualisation, Writing -original draft, Writing -review & editing Elie A Akl: Conceptualisation, Writing -original draft, Writing -review & editing Matthew J Page: Writing -original draft, Writing -review & editing Vivian A Welch: Writing -original draft Working Group, a voluntary collaboration of stakeholders interested in the application of Open Science principles in evidence synthesis conduct and publication.