key: cord-0075947-6i3y0zfk authors: Kowalczyk, Olivia S.; Lautarescu, Alexandra; Blok, Elisabet; Dall’Aglio, Lorenza; Westwood, Samuel J. title: What senior academics can do to support reproducible and open research: a short, three-step guide date: 2022-03-22 journal: BMC Res Notes DOI: 10.1186/s13104-022-05999-0 sha: 7ce3d368040a4990e5b1ceec0c468f970dfdbaf3 doc_id: 75947 cord_uid: 6i3y0zfk Increasingly, policies are being introduced to reward and recognise open research practices, while the adoption of such practices into research routines is being facilitated by many grassroots initiatives. However, despite this widespread endorsement and support, as well as various efforts led by early career researchers, open research is yet to be widely adopted. For open research to become the norm, initiatives should engage academics from all career stages, particularly senior academics (namely senior lecturers, readers, professors) given their routine involvement in determining the quality of research. Senior academics, however, face unique challenges in implementing policy changes and supporting grassroots initiatives. Given that—like all researchers—senior academics are motivated by self-interest, this paper lays out three feasible steps that senior academics can take to improve the quality and productivity of their research, that also serve to engender open research. These steps include changing (a) hiring criteria, (b) how scholarly outputs are credited, and (c) how we fund and publish in line with open research principles. The guidance we provide is accompanied by material for further reading. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s13104-022-05999-0. Increasing evidence shows that research in the biomedical and social sciences and research more broadly is difficult to replicate and/or reproduce [1] [2] [3] [4] [5] . One of the causes of this 'replication crisis' is thought to be misplaced incentives that can undermine research quality. For instance, publishers and funders generally give a selective advantage to novel or statistically significant results, thereby devaluing efforts to confirm published research [6, 7] . Further, employment evaluation criteria unduly focus on individual achievement, publication track records, and grant funding acquisition, which can hamper data sharing and collegiality while incentivising publishing in quantity at the cost to quality [8] [9] [10] [11] . Many and varied changes in policies and procedures are seeking to realign incentives to reward transparent, accessible, and reproducible research [12] [13] [14] , while grassroots initiatives are removing barriers to entry in learning and adopting best research practice [1, [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] . However, despite significant support, widespread adoption of open and reproducible research remains elusive [25] [26] [27] [28] . Further, there is little attention paid to how the current research culture contributes to bullying, harassment, mental health, and the resulting rising tide of researchers leaving academia [29] . For open research to become the norm, further engagement and support must come from senior academics given their routine involvement in supervision, peer review, journal editing, hiring, and informing institutional policies. Senior academics are, however, presented with unique social and practical barriers. For example, setting higher quality standards for junior researchers can be negatively perceived as 'ladder pulling' [30] , while the widely held perception that open research can stifle innovation or long-held academic freedoms can make researchers at all career stages hesitant to change current practices [27, [31] [32] [33] . Further, applying for grants [34] [35] [36] and teaching [37] occupy an increasing amount of work time, which means attending training, developing open research practices, or changing long-standing research routines can be costly and therefore deprioritized. Finally, the increasing literature on how to adopt open research is fast becoming overwhelming, contradictory, and mainly tailored to early career researchers [18, 23, 25, 26] . Therefore, we present a short guide highlighting three easy steps to introduce open research ideas and practices into existing research routines while avoiding the barriers mentioned above. These steps include (1) modifying hiring criteria, (2) crediting scholarly outputs with the contributorship model, and (3) securing grant funding and publishing in line with open research. Following the lead of similar initiatives, these steps are designed to appeal to the self-interests of researchers to motivate their engagement with open research practices [23, 38, 39] , with a unique focus on the viewpoint of senior academics. This is supplemented by materials for further reading. Step 1: Change how you hire Evidence shows that open research practices confer a competitive advantage in publishing scholarly outputs and acquiring grant funding (see Table 1 ), meaning that individuals with open research expertise are a desirable asset in lab groups or departments. However, such individuals will likely be missed in hiring and promotion opportunities as a result of the undue weight given to evaluation metrics such as h-indices and journal impact factors [9, 40] . Further, as open research is rarely mentioned in job descriptions, sought-after candidates cannot easily identify potential employers that value open research. Therefore, we encourage senior academics (where possible) to modify their hiring criteria to incorporate open research practices that support research quality and productivity. Modelled on a crowd-sourced initiative [41] , one feasible approach is to modify desirable/essential person specification criteria to include a track record of one or more open research practices (e.g., open data, open materials/code, pre-registration, open access publication, publishing preprints, and/or open peer review; see Table 1 for definitions). Criteria should be stated clearly and publicly in advertised job descriptions and/or hiring policies, while decisions about which open research practices to include should be made in consultation with faculties/departments to avoid unnecessarily disadvantaging staff/students. For example, where a track record of open access publications is not expected (e.g., for a PhD student/postdoctoral researcher), proxies for productivity or keen engagement in open research can include preprints, open materials, or open peer review. Instructive examples of how this can be achieved can be found here [42] and in our Additional file 1: Table S1 . Step 2: Change authorship to contributorship The main currency for career progression is authorship on scholarly outputs [11, 61, 62] . As a result, authorship disputes are widespread, leading to delays in submissions, conflicts among collaborators and journal editors [63] [64] [65] , and/or retractions [66] [67] [68] . Such intense competition over credit for scholarly outputs has significantly disadvantaged those in more precarious positions (such as black and minority ethnic groups, individuals on fixedterm contracts, and women), with 40% of early-career researchers reporting that credit for their work was given to other academics or research staff [29, 69, 70] . As large collaborative projects become the norm, contributions will be more difficult to dissect and therefore authorshiprelated issues will become more common [71] [72] [73] [74] . Issues with assigning credit for scholarly outputs are in part due to the lack of consensus-based and comprehensive standards. The commonly used standard, the International Committee of Medical Journal Editors (ICMJE; or the Vancouver guidelines), stipulates that authorship is contingent on substantive contributions (e.g., conceptual design, data collection, analysis, or interpretation, drafting and/or revising a manuscript) [75] . Still, ICMJE offers no adequate guidance on contentious issues, such as designating first, last, or corresponding authorship; assigning responsibility for the research; or dealing with large collaborations or other contributions (such as from librarians and statisticians) [76, 77] . These issues can be avoided with contributorship models of authorship, such as the Contributor Roles Taxonomy (CRediT), a consensus-based classification system that distinguishes 14 contributor roles (see Additional file 1: Table S2 ) that is now adopted in the submission process at leading publishers (e.g., Elsevier, PLoS, Wiley, and Springer) and hundreds of journals [78, 79] . CRediT documents individual contributions to a scholarly output in a standardised, accessible, and discoverable manner. This can be done at any stage in a research project, although the earlier the better to manage expectations of team members and to minimise Table 1 Open research practices and the career benefits they confer. Definitions are lifted from [43] Open research practice Definition Open Access Publishing A scholarly output accessible to the public free of charge. This can include green, gold or platinum/diamond forms of open access. Open access can be applied to the following scholarly outputs: peer-reviewed journal articles, conference papers, theses, book chapters, monographs, and images Publishing via open access is associated with higher citation rates and improves the speed and breadth of dissemination of scholarly outputs [44, 45] Open Data Publicly accessible, digitally-shareable data that are necessary to reproduce the reported results Facilitates collaboration [46] ; increases efficiency and sustainability [47] ; published papers linked with open data and/or materials are associated with a higher citation rate on average [23, 45, 48] ; when published with a digital object identifier (DOI), open data and/or materials can be a citable publication [49] ; synthetic datasets can help cross-validate analysis and improve reproducibility of analysis workflows [50] Open Materials Publicly available components of the research methodology needed to reproduce the reported procedure and analysis (e.g., code, software, workflows, etc.) Open Peer Review A findable, freely and publicly accessible, and signed peer review either pre-or postpublication Academics who act as reviewers can get credit for their work [51] Preprints Complete, non-peer-reviewed manuscript entered in a time-stamped and publicly accessible location, usually an institutional or disciplinary repository (e.g., PsyArXiv, LawArXiv, UCL Press, MedrXiv). Preprints are often also submitted for peer review and publication in a traditional scholarly journal, but this is not mandatory Wider, faster, and cheaper dissemination of research [52] ; greater opportunity for feedback outside of formal peer-review [24] ; posting a manuscript as a preprint before formal publication can increase citations and impact [53, 54] ; improves chances of publication in journals with high impact factors [55] Preregistration A publicly available time-stamped study design and/or analysis plan that is registered in an institutional registration system (e.g., ClinicalTrials.gov, Open Science Framework, AEA Registry, EGAP) Boost a researcher's reputation [56] ; preventative measure against post-hoc critique (i.e., CARKing-critiquing after the results are known) during peer-review [39, 57, 58] ; prospective registration of a study design can be a citable publication; comply with submissions guidelines set by International Committee of Medical Journal Editors (ICMJE) Registered Reports A peer-reviewed journal article where the decision to publish is based on a twostage peer-review process. First, following successful peer-review, a pre-specified study and/or analysis protocol is accepted in principle by a participating journal before data has been collected or accessed. Second, providing the authors closely followed the protocol and successful peer-review, the final manuscript is published regardless of the results Guaranteed publication regardless of study results, providing the registered protocol and/or analysis is followed [59] ; reduces CARKing [39, 57, 58] ; cited at comparable or slightly higher levels than conventional peer-reviewed articles [60] ; stage one peerreview provides additional peer-review feedback future authorship issues. The web-based application, Tenzing, automates this process and produces a CRediTcompatible manuscript for publication [80] . Although the contributor roles are fixed, their definitions can be customised to a particular research discipline for clarity. Further, CRediT can provide a useful framework for deciding on authorship designation. For instance, the degree of contribution can be specified as 'lead' , 'equal' , or 'supporting' , which can inform authorship order [71, 73] . Moreover, contributions to 'data curation' , 'project administration' , and 'validation' can instruct who should be the corresponding author. CRediT also offers unique opportunities to improve productivity, particularly in terms of fostering collaborations, by signalling the expertise of members of your research group, recognising individual contributions to large teams, and acknowledging roles which tend to be overlooked despite providing valuable insight or support (e.g., project administration). See Additional file 1: Table S3 . Funders and journals are beginning to advantage open research practices with novel initiatives and policy changes. Thus, to be in a position of strength, senior academics should engage with open research in seeking funding and publishing their research outputs. Funders and journals widely endorse the practice of making sure that research data should be 'as open as possible, as closed as necessary' , with new policies being introduced to further compliance with this practice [81] . Most funders now also require a data management plan (i.e., a detailed specification of how data or materials will be curated, shared, or used) as standard [82] . Data availability statements, indicating where data and materials are available or specifying reasons for exemptions from datasharing, are also compulsory for submissions to a growing number of journals, including Science, Nature, and the BMJ [83] [84] [85] . Data can also be archived and shared through data journals (such as, [84] [85] [86] [87] [88] [89] [90] ) or in third-party repositories (e.g., GitHub, Open Science Framework, and Zenodo), which allow control over how data and code are used and shared by assigning licences and DOIs [1, 49, 93] (See Additional file 1: Table S4 ). Perhaps the most significant and less well-known policy changes concern preprints, which encourage the publication of scholarly outputs in a faster, more impactful, and more accessible manner. A preprint is a timestamped, non-peer reviewed manuscript made freely and publicly accessible via an online server typically within 72-h of submission (e.g., PsyArXiv, LawArXiv). Thus, the significant time lag between manuscript submission and its publication (median days, 165) [94] and the infeasible journal open access fees [95] do not apply to preprints. Because of faster and wider dissemination, grantees are increasingly required to deposit preprints, particularly if funded research is of significant public health benefit (e.g., Bill & Melinda Gates Foundation) [96] . Further, a majority of journals permit preprints to be shared before or during manuscript submission [97] (Additional file 1: Table S4 ), presumably due to evidence that journal articles linked to preprints have higher citation rates [53, 55] . Influential journals (e.g., BMJ, The Lancet) and funders (e.g., The National Institutes of Health, Wellcome Trust) are now explicitly stating that preprints can be cited [98, 99] . Preprints can additionally be referenced in researcher track records when applying for funding [96] and included in submissions to the UK Government funding organisation, the Research Excellence Framework [98] . The Table 2 and additionally curated a list of funding opportunities obtained by using data scraping, available at https:// loren zada. github. io/ openr esear ch_ fundi ng/. In this list, we selected funding opportunities mentioning keywords related to open research (e.g., replication study, reproducible code, preprint), after data scraping was performed from the NIH and UKRI funding websites. Of note, website selection for data scraping was based on whether automated data collection was permitted for a given website. For further information, please refer to the open code at https:// github. com/ Loren zaDA/ openr esear ch_ fundi ng. This list not only illustrates the mounting financial commitment to open research practices and projects from grant funders, but will hopefully encourage senior academics to apply for funding or for them to support applications from early career researchers in their research team. 'We create our culture, invisible though it may be, and we therefore have it collectively within ourselves to change our culture for the better' ( [118] , p. 92). Academic researchers typically aim to reach the highest standards of best research practice, but are hampered by perverse incentives and cultural norms. However, we sought to lower barriers of entry into open research for senior academics, and to highlight that open research is advantageous for research grant capture, productivity, and integrity. More remains to be done, but our short, easy-to-follow, three-step guide will hopefully mark the first steps into a wider adoption of open research for many senior academics. The online version contains supplementary material available at https:// doi. org/ 10. 1186/ s13104-022-05999-0. Additional file 1: Table S1 . Examples of open science practices in university policies for hiring and promotion. A manifesto for reproducible science Cancer reproducibility project releases first results Towards a more reproducible ecology. Ecography Estimating the reproducibility of psychological science Never waste a good crisis: confronting reproducibility in translational research Negative results are disappearing from most disciplines and countries The natural selection of bad science Measuring the prevalence of questionable research practices with incentives for truth telling Assessing scientists for hiring, promotion, and tenure The Hong Kong principles for assessing researchers: fostering research integrity Academic criteria for promotion and tenure in biomedical sciences faculties: cross sectional analysis of international sample of universities Promoting an open research culture A consensus-based transparency checklist Open Science challenges, benefits and tips in early career and beyond Grassroots training for reproducible science: a consortium-based approach to the empirical dissertation Seven easy steps to open science Data Skills for Reproducible Science How to become a Bayesian in eight easy steps: an annotated reading list Easing Into Open Science: A Guide for Graduate Students and Their Advisors A practical guide for transparency in psychological science How open science helps researchers succeed Hensel Z. On the value of preprints: an early career researcher perspective Attitudes toward Open Science and public data sharing: a survey among members of the German Psychological Society Motivating participation in open science by examining researcher incentives Data sharing in psychology: a survey on barriers and preconditions The transparency of quantitative empirical legal research What researchers think about the culture they work in The costs of reproducibility What drives academic data sharing? How do scientists define openness? Exploring the relationship between open science policies and research practice The Oncomouse that roared: hybrid exchange strategies as a source of distinction at the boundary of overlapping institutions Contest models highlight inherent inefficiencies of scientific funding competitions On the time spent preparing grant proposals: an observational study of Australian researchers To apply or not to apply: a survey analysis of grant writing costs and benefits Is paid research time a vanishing privilege for modern academics? [Internet]. Times Higher Education (THE) Five selfish reasons to work reproducibly Seven Selfish Reasons for Preregistration Recognition and reward in the academy: valuing publication oeuvres in biomedicine, economics and history Recognising Open Research Practices in Hiring Policies: Modular Certification Initiative Modular Certification Initiative Academic job offers that mentioned open science Open Science Framework. Badges to Acknowledge Open Practices The citation advantage of linking publications to research data The academic, economic and societal impacts of Open Access: an evidence-based review Ten simple rules to enable multi-site collaborations through data sharing Our path to better science in less time using open data science tools The state of OA: a large-scale analysis of the prevalence and impact of Open Access articles A data citation roadmap for scientific publishers A synthetic dataset primer for the biobehavioural sciences to promote reproducibility and hypothesis generation Ten considerations for open peer review. F1000Research [Internet] Preprints: An underutilized mechanism to accelerate outbreak science The effect of bioRxiv preprints on citations and altmetrics Releasing a preprint is associated with more attention and citations for the peer-reviewed article What bioRxiv's first 30,000 preprints reveal about biologists Pre-registration and registered reports: a primer from UKRN Registered reports are an ally to early career researchers Registered reports: a method to increase the credibility of published results What's next for registered reports? Evaluating Registered Reports: A Naturalistic Comparative Study of Article Impact Publication metrics and success on the academic job market Authors' opinions on publication in relation to annual performance assessment Resolving authorship disputes by mediation and arbitration What can be done to resolve academic authorship disputes? Science journal editors' views on publication ethics: results of an international survey Lab leaders must create open and safe spaces to improve research culture | Wellcome Lifelong learning: science professors need leadership training Some hard numbers on science's leadership problems Last place? The intersection of ethnicity, gender, and race in biomedical Credit where credit is due? Regulation, research integrity and the attribution of authorship in the health sciences How can we ensure visibility and diversity in research contributions? How the Contributor Role Taxonomy (CRediT) is helping the shift from authorship to contributorship Rethinking authorship in the era of collaborative research Beyond authorship: attribution, contribution, collaboration, and credit Law" and order Defining the Role of Authors and Contributors Contributorship, not authorship: use CRediT to indicate who did what Farewell authors, hello contributors CRediT -Contributor Roles Taxonomy Documenting contributions to scholarly articles using CRediT and tenzing Tenzing: documenting contributorship using CRediT A funder-imposed data publication requirement seldom inspired data sharing Digital Curation Centre. Overview of funders' data policies | DCC Public availability of published research data in high-impact journals Increasing value and reducing waste: addressing inaccessible research The new BMJ policy on sharing data from drug and device trials Scientific Data [Internet]. Nature Welcome to DataCite Figshare-credit for all your research The Dataverse Project-Dataverse.org The Dryad Digiti Repository Neurodata Without Borders-The Kavli Foundation Data sharing and how it can benefit your scientific career Same Time Next Year: crunching PubMed data Open access: the true cost of science publishing Funder policies Welcome to Sherpa Romeo -v2.sherpa Preprints are valid research outputs for REF2021 Transpose: A database of journal policies on peer review, co-reviewing, and preprinting Center for Open Science issues 29 grants to develop open tools and services to support scientific research Preregistration Challenge: Plan, Test Open Science (OS) Fund 2020/2021 | NWO Award -Einstein Foundation Berlin European Commission -European Commission Open Science Award | Organization for Human Brain Mapping Credibility in neuroscience to be championed through new BNA prize | News | The British Neuroscience Association Leamer-Rosenthal Prizes Nomination Process Berkeley Initiative for Transparency in the Social Sciences Seeking Projects at the Intersection of Openness and Science research data, including large and complex data types • gold Open Access which fosters wider collaboration and increased citations maximum visibility for your research: over 100M website views per year submit your research ? Choose BMC ER2)|Beta site for NSF -National Science Foundation Shuttleworth Foundation-Applications The Shuttleworth Foundation Open access funding and reporting Research Enrichment -Public Engagement | Grant Funding Wellcome Data Prizes Research culture and reproducibility Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations The author would like to thank those who provided helpful feedback on earlier versions of this manuscript, which include Marion Criaud and Sheut-Ling Lam.