key: cord-0284370-lzempp0o
authors: Christie, Alec P.; White, Thomas B.; Martin, Philip; Petrovan, Silviu O.; Bladon, Andrew J.; Bowkett, Andrew E.; Littlewood, Nick A.; Mupepele, Anne-Christine; Rocha, Ricardo; Sainsbury, Katherine A.; Smith, Rebecca K.; Taylor, Nigel G.; Sutherland, William J.
title: Reducing publication delay to improve the efficiency and impact of conservation science
date: 2021-04-01
journal: bioRxiv
DOI: 10.1101/2021.03.30.437223
sha: 0c11193b362eaccfbe472eb796664761c7552a1a
doc_id: 284370
cord_uid: lzempp0o

Evidence-based decision making is most effective with comprehensive access to scientific studies. If studies face delays or barriers to being published, the useful information they contain may not reach decision-makers in a timely manner. This represents a potential problem for mission-oriented disciplines where access to the latest data is paramount to ensure effective actions are deployed. We sought to analyse the severity of publication delay in conservation science — a field that requires urgent action to prevent the loss of biodiversity. We used the Conservation Evidence database to assess the length of publication delay (time from finishing data collection to publication) in the literature that tests the effectiveness of conservation interventions. From 7,415 peer-reviewed and non-peer-reviewed studies of conservation interventions published over eleven decades, we find that the mean publication delay (time from completing data collection to publication) was 3.6 years and varied by conservation subject — a smaller delay was observed for studies focussed on the management of captive animals. Publication delay was significantly smaller for studies in the non-journal literature (typically non-peer-reviewed) compared to studies published in scientific journals. Although we found publication delay has marginally increased over time (1912-2020), this change was weak post-1980s. Publication delay also varied inconsistently between studies on species with different IUCN Red List statuses and there was little evidence that studies on more threatened species were subject to a smaller delay. We discuss the possible drivers of publication delay and present suggestions for scientists, funders, publishers, and practitioners to reduce the time taken to publish studies. Although our recommendations are aimed at conservation science, they are highly relevant to other mission-driven disciplines where the rapid dissemination of scientific findings is important.

Introduction 56 Across many mission-oriented disciplines, where there is an urgent need to tackle a societal 57 issue, evidence-based decision making is critical to improving the effectiveness and 58 efficiency of practice. This requires comprehensive access to scientific studies providing 59 data useful for judging the likely effectiveness of actions. New scientific studies not only add 60

to the relevant corpus of information that can guide decisions, but are likely to be particularly 61 influential due to continually evolving technologies, methodologies, and skills, as well as the 62

focus on issues of current concern. However, if new studies are not made available, or 63 delayed in being so, relevant information useful for decision making (i.e., evidence; Salafsky 64 et al. 2019) cannot easily be located by decision-makers in a timely manner. Not having 65 rapid access to evidence to inform decision making risks suboptimal outcomes. 66 Biodiversity conservation is an example of a mission-oriented discipline and is motivated by reviews as we were interested in the publication delay of primary literature. We focused on 134 the number of unique studies of an intervention within each Conservation Evidence 135 synopsis. For example, if a publication reports studies of two different interventions (e.g., 136

supplementary feeding and provision of artificial nests), then these studies are counted 137 separately. Using this classification of conceptually distinct studies, we were able to extract 138 information on when 7,415 studies were published and when their data collection ended. 139

Approximately 3% of all studies did not report dates (280 out of 8,115 in the entire database 140 as of December 2020) and so were excluded from the analyses. 141

Using the name of each study's publication source and a dataset downloaded from SCImago 142 (2020), we categorised the literature in which studies were published into three groups: i.) a 143 'recognised journal' (which had SCImago (2020) impact metricstypically peer-reviewed 144 journals); ii.) in 'unrecognised journals' (which did not have SCImago impact metrics -145 typically a mix of less conventional journals that may lack peer-review); and iii.) the 'non-146 journal literature' (often termed 'grey literature', which typically lacks peer review). This 147 three-way separation of publication sources is a crude proxy for whether they are likely to be 148 peer-reviewed (recognised journals = high; unrecognised journals = low-medium; non-149 journal literature = low)thereby enabling some approximate estimation of the time taken 150 for peer-review. 151

Where names of publication sources did not match the names given within the SCImago 152 dataset, we manually searched to check whether names had changed over time, or had 153 been incorrectly recorded in the Conservation Evidence database, and allocated publication 154 sources to the 'recognised journal' category if a match was then found. Where names still 155 did not match the SCImago dataset, we classified whether publication sources were either 156 'unrecognised journals' or from the 'non-journal literature' by manually reviewing the online scientific journal was categorised as an 'unrecognised journal'. For names of all publication 162 sources in each of the three publication categories, see Tables S1-S3 . 163

We extracted temporal information from the Conservation Evidence database (publication 164

year) and a summary of each study that included information on when the study was 165 conducted (the years the study began and ended). We defined the end year of a study as 166 the year within which data collection ended (not when the intervention ended, for example). 167

End years were extracted from Conservation Evidence summaries using regular expressions 168 and text mining of the website (www.conservationevidence.com) with the XML package 169 (Lang 2020a) and RCurl package (Lang 2020b) in R statistical software version 3.5.1 (R 170

Core Team 2020). This extraction was necessary because this information is currently not in 171 the database. We checked the accuracy of text mining by reviewing data for 79 studies 172 (approximately 1% of the total number of studies analysed) and found that 94% had the 173 correct study end year. Although there were a small number of errors, these were mostly 174 caused by assigning the publication year as the study end year, and therefore would yield an 175 underestimate of publication delay. In addition, automating the extraction of dates from study 176 summaries offered the most feasible and reproducible way to analyse the entire evidence 177 base, and avoided human error and unconscious bias that would affect manual extraction of 178 dates (Christie et al. 2020 (Christie et al. , 2021 . 179

To determine publication delay, we subtracted the end year of each study from its 180 Table S4 . Therefore, we used three explanatory variables (synopsis, publication 198 source, and publication year) to predict the response variable of publication delay. After an 199 initial Poisson GLM revealed an overdispersion parameter value of 2.57, we used a quasi- Hothorn et al. 2008) to test for significant differences between synopses and between 207 publication sources using our GLM. For the purposes of our visualisations, the plotted mean 208 publication delay (and 95% Confidence Intervals) across all synopses was obtained using an 209 intercept-only GLM. We used GLMs with a single fixed effect to plot summary statistics for 210 visualisations of different synopses, different publication sources, and changes over time 211

(i.e., using a fixed effect of either synopsis, publication source, or publication year, 212 respectively). 213 (which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission. As more recently published studies may be more likely to suffer from a longer delay, we 214 conducted a sensitivity analysis by restricting the data we analysed in our original GLM to 215 1980-2020, 1990-2020, and 2000-2020 (Table S8 )we discuss these results later in the 216

In a separate analysis, we tested for significant differences in publication delay between 218 studies testing interventions on species with different IUCN Red List statuses. To do this we 219 extracted data from the Conservation Evidence database on the species studied within taxa- Endangered) of all species for each published study. For example, if a study targeted 232 multiple species, such as two that were listed as Least Concern and one listed as 233

Endangered, we considered that as a study on an Endangered species. There were 234 insufficient studies on species with IUCN Red List statuses of Data Deficient (zero studies) 235 or Extinct in the Wild (less than eight studies) and so we did not include these categories in 236 our taxonomic analyses. We were unable to obtain the IUCN Red List status of species at 237 the time when the study was conducted and therefore had to use the current status of 238 species in the latest IUCN (2020) Red List update. Whilst this may mean that, for some 239 studies, certain species may have changed in their Red List status in the intervening years, 240 (which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission. 

There was a small, but statistically significant, increase in publication delay from 1912 to 279 2020 ( Fig. 3; t=3 .598; p<0.001; Table S5 ); based on sensitivity analyses, a statistically 280 (which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission. Table S5 for full model result).

290 all data were used in the GLM (see full data figure Fig.S1 ). We conducted sensitivity analyses 291 to check whether the trend changed in more recent decades (see Table S8 ).

(which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission. 

(which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission. Our results suggest that conservation decision-makers must typically wait, on average, 3.5 324 years for the latest evidence testing the effectiveness of conservation interventions to be 325

published. There were significant differences in publication delay between conservation 326 subjects (synopses) and between publication sources, where studies testing interventions on 327 captive animals and studies from non-journal literature had a smaller delay. Although we 328

found publication delay has marginally increased over time (1912-2020), sensitivity analyses 329 In conservation, a great deal can happen in 3.5 years -a species' population may drastically 341 decline, new threats may emergeand conservationists may have to take rapid action to 342 avert biodiversity and habitat loss. Without faster access to evidence on effectiveness, there 343 is a risk that practitioners pursue ineffective practices and mis-allocate conservation 344 resources at a time when we cannot afford to do so. Our findings are particularly concerning 345

given that we used a conservative approach to estimate publication delay by coarsely 346 quantifying publication delay using differences between years (which rounds down 347 publication delay to zero for any studies completed and published within a calendar year). 348 (which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission. are easier to plan, conduct, and write-up. It may also be that authors target a relatively 353 narrow pool of specialist journals for publication. This could mean that rejection and 354 resubmission to another journal, for reasons other than the quality of science, is less 355 common. It is also possible that specific journals focused on captive animals have faster 356 publication times. Ultimately, it is likely that a combination of reduced time to submission and 357 faster journal publication processes have led to this smaller delay. 358

To better understand and minimise publication delay in scientific journals, it is useful to tease 359 apart the potential sources of delay, namely: (1) 'write-up delay' (the time taken from 360 finishing data collection to submitting a study to the first journal); (2) 'resubmission delay' 361 given we used differences between years as our metric of delay. 388 (which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission.

The copyright holder for this preprint this version posted March 31, 2021. ; https://doi.org/10.1101/2021.03.30.437223 doi: bioRxiv preprint destination journal delay had reduced significantly in conservation from 572 days to 402 390 days, a faster decrease than in other fields they studiedalthough this delay is still 391 substantial, and will hamper effective and timely conservation action. Whilst we did not find 392 any significant decreases in publication delay over the time periods we analysed (including 393 sensitivity analyses on more recent decades), the greater likelihood for more recent studies 394

to have a longer delay could have masked progress in reducing publication delay to some 395 extentalthough we would argue it is unlikely that publication delay has decreased 396 substantially, if at all. Furthermore, we found a significantly longer overall publication delay 397 for studies published in journals (mean of ~2.7 years for recognised journals and ~2.7 for 398 unrecognised journals) than in the non-journal literature (mean of ~2.2 years), suggesting 399 that features typically associated with journal publication, such as peer review and editorial 400 processes, are still contributing substantially to publication delay. Therefore, further 401 improvements to the peer-review and publication process could reduce some of the delay 402 we have observed. 403

A more systemic problem, however, is likely to be the combination of write-up delay and 404 resubmission delay, which are collectively known as 'submission delay' (Fig. 1) compared to ~3.7 years for recognised journals). Previous studies have included this 443 resubmission delay within submission delay (see Fig. 6 ), but our findings tentatively suggest 444 that write-up delay is generally smaller than the combination of resubmission delay and 445 destination journal delay. We suggest future work could build on our results by directly 446 quantifying and disentangling the components of publication delay observed, to help target 447 action in areas that require more focus. 448

In Table 1 , we present a set of possible solutions that could help to reduce write-up, 449 resubmission, and destination journal delay. Whilst the solutions outlined in Table 1 are 450 focussed specifically on conservation science, we believe they are relevant to many different 451 disciplines tackling publication delay. The COVID-19 pandemic has seen a far-reaching 452 response from the scientific community to boost the rate at which scientific research is being 453 conducted and published (including studies of healthcare interventions) through clear 454 incentives to publish, rapid peer-review, and streamlined editorial processes (Horbach 455 2020). 456

We believe that the conservation community could learn from this effort to build a strong 457 evidence base that is rapidly updated with the latest studies of conservation interventions to 458 help address the biodiversity crisis. Nevertheless, there is concern over the unavoidable 459 trade-off between speed and quality in the dissemination of scientific evidence; for example, decision making in any mission-driven discipline, but particularly in biodiversity conservation 471

given the need to reverse the dramatic loss of biodiversity. Concerted action is required to 472 streamline the rigorous testing and reporting of conservation interventions' effectiveness to 473 cover known gaps and biases in the evidence base (Christie et al. 2020 (Christie et al. , 2021 . We believe 474 our study clearly demonstrates the need for academics, practitioners, journals, 475 organisations, and funders to work together as a scientific community to reduce publication 476 delay as much as possible. 477 (which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission. • Less strict formatting and structure requirements for initial submission. • Journals agree a set of styles to reduce the cost of resubmission. • Journals produce article templates (e.g., as

for Conservation Evidence Journal and Oryx). 

use other's reviews to guide their decisions.

papers in Biological Conservation; Biological Conservation 2021) and/or send drafts to reviewers pre-review to allow reviewers to prepare comments (Sutherland & Lythgoe 2020). • Once accepted, publish quickly (e.g., early view, online publishing) to reduce the time spent in publication limbo. • Embrace new initiatives of transparent peerreview to: share reviews between potential publishers, identify papers of interest and quickly publish the already reviewed articles. • Move to a peer-review system that is "publish first, curate second'' through strengthening and increasing the use of preprint servers, allowing open, transparent peer-review, and the development of curation journals to select those articles of interest for specific audiences (Stern & O'Shea 2019). This has been realised during the COVID-19 pandemic with the creation of RR:C19 a journal that rapidly and transparently reviews and curates pre-prints (Dhar & Brand 2020).

Destination journal delay

• Select platforms and journals that have taken steps to reduce publication delay in the publication and peer-review process. • Consider the use of submission models (such as 'Peer Community In' or 'Octopus') that provide transparent peer-review and recommendation of pre-prints or initial submissions, but without the requirement for, although compatible with, journal publication.

• Promote the use of platforms and journals that have taken steps to reduce publication delay in the publication and peer-review process.

Buck: Rejection Rates and Impact Factors in Ecological Journals

Biological Conservation. 2021. Guide for Authors

The publishing delay in scholarly peer-reviewed journals

Poor availability of context-specific 491 evidence hampers decision-making in conservation

The challenge of biased evidence in 494 conservation

Coronavirus: time to re-imagine academic publishing

Methodological quality of COVID-19 clinical research

A Severe Lack of Evidence Limits Effective Conservation of the World's 509 Primates

Slow-moving journals hinder conservation 511 efforts

XML: Tools for Parsing and Generating XML Within R and S-Plus

Client Interface for R. R package 515 version 1.98-1-2

Bending the curve of terrestrial biodiversity needs an integrated 517 strategy

Aiming higher to bend the curve of biodiversity loss

Making conservation science more reliable with 531 preregistration and registered reports

Does it take too long to publish research?

Doing more good than harm--Building an evidence-base for 534 conservation and environmental management

Delays in the 536 publication of important clinical trial findings in oncology

R: A language and environment for statistical computing. R Foundation 539 for Statistical Computing

Defining and using evidence in conservation practice

A proposal for the future of scientific publishing in the life

At the same time, writing-up and publishing studies of interventions is not easy. Even after 427 write-up, a manuscript may be rejected from several journals, including for subjective 428 reasons of the perceived level of interest from readers rather than the strength of results or 429 their importance for conservation. Substantial edits are then required to suit different 430 journals' formats, and reviews may suggest major changes which take time and resources to 431 implement. It is common for published manuscripts to have gone through an iterative 432

• Authors "calibrate" submissions to journals best suited to their work (Vosshall 2012) to avoid lengthy rejections and resubmissions. • Authors publish pre-prints online (e.g., BioRxiv, EcoEvoRxiv, SocRxiv) when the work has been submitted to a journal. However, caution should be taken if disseminating results due to the lack of peer-review. • Authors pre-register study designs and/or analyses before undertaking data collection where possible (or submit a registered report). This could help reduce the likelihood of rejection and the need for lengthy resubmissions and revisions due to poor quality study design or analyses (Parker et al. 2019 ). • Adoption of one submission models (e.g., 'Peerage of Science' and 'Peer Community In') that provide peer-review that multiple publishers can access, and link papers with interested journals who• Reduce unnecessary effort required for initial submissionse.g., universal formatting, word counts, flexibility in section layouts, presubmission enquiries etc.