key: cord-0051327-1ioas6yd authors: Hobday, Alistair J; Browman, Howard I; Bograd, Steven J title: Publishing and peer reviewing as indicators of the impact of COVID-19 on the productivity of the aquatic science community date: 2020-09-14 journal: ICES J Mar Sci DOI: 10.1093/icesjms/fsaa151 sha: d411e91b49f9ff263ed5aa72c9841048a759632d doc_id: 51327 cord_uid: 1ioas6yd Beginning in February 2020, COVID-19-related stay at home orders and workplace shutdowns worldwide have disrupted personal and professional lives, including those of aquatic scientists. Manuscript submission and peer reviewing data from journals may be indicators of productivity impacts among aquatic scientists. We tested four null hypotheses: the COVID-19 disruption has had no effect on (i) the number of submissions to journals, or (ii) the geographic region in which the corresponding author is based, nor on the peer review process in terms of (iii) acceptance rate of requests to review and (iv) time in review. We used data provided by seven leading aquatic science journals covering the period 2009–2020 and representing 32 756 submissions. Submission differences varied between journals and were lower than expected in March 2020, but due to increases in subsequent months, there was no overall change in the number of submissions during the COVID-19 disruption months of February–June 2020. Geographic patterns in the number of submissions varied more by journal than by region, with both higher and lower numbers of submissions relative to expected numbers. Acceptance rates of requests to review were ∼2% lower overall; however, time in review declined by an average of 5 days relative to earlier years, showing that those scientists undertaking reviews did them more quickly during the COVID-19 disruption. Collectively, these results show that the overall productivity of the aquatic science community, as measured by publications and reviewing rates and times, has thus far only been slightly disrupted, although the impacts will vary greatly among individuals depending on life circumstances. The breadth and longevity of this disruption are unprecedented, making it important to continue to assess the relative impacts across a wide demographic range of aquatic scientists and to consider approaches to allow those differentially affected to recover to pre-COVID-19 levels of productivity. An extreme event is a dynamic occurrence within a limited timeframe that impedes the normal functioning of a system or systems (Broska et al., 2020) . Scientists study the impact of extreme events to help society build resilience and minimize impacts in the future (Altwegg et al., 2017; Solow, 2017) or to better understand the full range of system dynamics (Broska et al., 2020) . Scientists are also affected by extreme events yet are not often themselves the subject of study. Extreme events have always affected individuals and are explicitly recognized as mitigating circumstances in grant proposal schemes or in considering "output" relative to opportunity, with allowances provided for family responsibilities in some countries (Malisch et al., 2020; Myers et al., 2020) . These individual disruptions (or life choices, in the case of parenting) contrast with the collective extreme event being experienced by societies all around the world due to the COVID-19 pandemic. Scientists are considered one of the more resilient segments of the work force, in terms of maintaining employment, but in many countries have still been affected due to social distancing requirements that have closed or reduced office and laboratory use, curtailed travel for activities such as project work and conferences, and halted research voyages on ships and small boats, and field and laboratory work in general. The extremes of the spectrum of anticipated COVID-19-related impacts for scientists who have retained employment are "this is going to be a productive period" (e.g. Fleming, 2020) vs. "this is going to be a very big disruption to productivity" (e.g. Ling, 2020) . Anecdotal evidence of impacts on early career researchers (Pain, 2020) , parents (Staniscuaski et al., 2020; 500 Women Scientists, 2020) , and women (Minello, 2020; Vincent-Lamarre et al., 2020) suggests declines in productivity as a result of COVID-19 are likely (https://www.thelily.com/women-academics-seem-to-be-submittingfewer-papers-during-coronavirus-never-seen-anything-like-it-saysone-editor/; https://www.insidehighered.com/news/2020/04/21/ early-journal-submission-data-suggest-covid-19-tanking-womensresearch-productivity). Part-time volunteer editors may also have additional responsibilities that may affect manuscript handling times (https://publicationethics.org/news/letter-cope-chair-april-2020). Cessation of day care and school in many countries has required parents to increase caring responsibilities while also maintaining various levels of work-related activity (Staniscuaski et al., 2020) , which may have been intensified by an increase in the number of virtual meetings. For some researchers, time savings, and increased production of publications, might be a benefit of reduced travel. In one of our organizations, for example a group of 90 marine scientists went from spending >300 person days per month travelling for work to 0 days for the months of March-June 2020. Field work, including for research voyages, has also ceased in many locations. While some of this time could still be directed to completing the same work-related tasks remotely, this represents considerable potential savings in time previously spent travelling. Grant submission deadlines have been delayed in many countries and even cancelled, which might lead to an increased focus on the production of manuscripts and on undertaking review assignments. We evaluate potential COVID-19-related trends in researcher productivity in aquatic science during the first half of 2020. While productivity may take a range of forms, we use the evidence base available from one important area of activity that aquatic scientists undertake-submission of manuscripts to journals and review of manuscripts submitted to these journals-as a proxy for productivity. We test four null hypotheses to assess for changes in these proxies of productivity during the COVID-19 disruption period. (1) There is no change in the number of manuscripts being submitted. (2) There is no change in the geographic pattern of submissions, based on the country of the corresponding author. (3) There is no change in acceptance (or completion) rate for review requests. (4) There is no change in the time taken to complete reviews. While gender differences in scientific publication submissions have been posited (e.g. Vincent-Lamarre et al., 2020; Staniscuaski et al., 2020) , with the data collected by journals, and the form in which data were shared with us, it was unfortunately not possible to investigate this aspect. Two datasets were assembled to assess the productivity of aquatic scientists associated with publishing in peer-reviewed journals. The first dataset assembled information on article submission and reviewing rate provided by seven aquatic science journals. We selected journals that had been publishing papers for at least the last decade to have a sufficient reference period. The seven journals were Canadian Journal of Fisheries and Aquatic Sciences, Fisheries Oceanography, Fisheries Research, ICES Journal of Marine Science, Limnology and Oceanography, Marine and Freshwater Research, and Marine Ecology Progress Series. Since the data provided by these journals and their publishers are confidential, they are presented in an anonymized manner. Changes in editorial software systems over time meant that some data were not consistently collected over the entire time period-when that was the case, those data were removed from analyses for that particular journal (e.g. time in review data was inconsistently archived). The second data set was derived from employee submission records for intended peer-reviewed articles from the national Australian science agency (CSIRO) and by a regional division of the United States National Marine Fisheries Service (NOAA/ NMFS). Both these government organizations require employees to register all manuscripts prior to submission and, while not specific to any particular journal, these data can be used as another proxy for productivity. For each journal, we received data on the number of submissions per month and compared February-June 2020 with data from the same months in previous years (e.g. February-June 2009-2019) (Hypothesis 1). We first checked for trends over time that could confound detection of a COVID-19 effect. In the case of trends (e.g. increase or decrease in submissions over time), we calculated the expected number of submissions based on a linear trend for monthly data prior to 2020 and calculated the difference between the observed and expected values for each of the months February-June 2020. Where there was no trend over time, we subtracted the number of submissions for each 2020 month (February-June) from the average number of submissions in that month prior to 2020 (e.g. February 2020 À the average of February 2009-2019). Negative values indicate fewer than expected submissions, and vice versa. Country of origin for the corresponding author (most often the first author) for submissions was also provided (Hypothesis 2). We assume this was where the author resided. Geographic patterns in submissions were considered at a regional scale by allocating the corresponding author country to a region (North America, Europe, Australia/Pacific, Asia, Africa, and South America), as submissions for individual countries were too few to detect changes for the COVID-19 period. As there were trends over time for most region-journal combinations, we calculated the difference between the observed and expected numbers of submissions for each region in 2020 compared to the expected number based on the linear trend prior to 2020, as for Hypothesis 1. In seeking reviewers for a submitted paper, an editor will issue individual requests for each paper (P), which are accepted (A), declined, or ignored at some rate (%A). Sometime later (t2), a percentage of these accepted reviews are completed and returned (%C), and a decision (D) is then made some days later by an editor (t3). Data provided on this process differed by journal, depending on their record-keeping system and were variously based on the number or percentage of reviews accepted or completed (divided by the number of invitations issued), per paper, per month, or per year. We used only data on reviews accepted (A) or completed (C) for original submissions and not revised manuscripts, and hereafter use the general term "review acceptance rate" to describe this proxy. As annual data were provided by some journals, inclusion of January 2020, before most of the isolation measures were implemented around the world, is expected to reduce any COVID-19 effect. Individual or monthly data were aggregated to an annual review acceptance rate to facilitate comparisons by journal, where the year 2020 (i.e. January-June) was assumed to encompass the COVID-19 effect (Hypothesis 3). For two journals that provided monthly data, there were no differences between review acceptance rates for the period January-June in pre-2020 years, and the whole year. For each journal, we calculated the anomaly in review acceptances using the form of the data provided-either the review acceptances (%A) or completions (%C) data for 2020. Several journals also provided data on the "time in review"-based on the time that the first (or all) reviewers took to complete (t2) their review(s) of a submission (Hypothesis 4). One journal also provided data on the time between an editor sending a review request and the acceptance by the reviewer [t1 in (1)]. Data on review acceptance rate and time in review were both inspected for temporal trends and, if necessary, detrended for each journal before calculating differences (anomalies) from expected values based on the trend. The CSIRO (Australia) requires employees (lead authors and junior authors for submissions not led by CSIRO authors) to register submissions to peer-reviewed journals. These data allow (i) a test of marine science manuscript submissions across a range of journals and (ii) a comparison of the relative productivity change in 450 marine scientists employed by CSIRO with the 4000 scientists employed across other research domains in CSIRO. As for Dataset 1, the number of submissions per month was determined for the Marine Division, and across all other divisions for the period 2015 to May 2020. Data were detrended if necessary and submissions for 2020 were compared to the period 2015-2019. Similarly, the NOAA/NMFS (USA) requires submissions to be approved by a senior manager. Annual data for the period January-May 2015-2020 were provided. The anomaly from the annual trend was calculated for each year, and then 2020 compared to the period 2015-2019. The submission dataset was based on 32 756 submissions to seven journals spanning the period January 2009-June 2020 ( Table 1) . Four of the submission time series were detrended (see Supplementary material). The number of submissions for the months of February-June 2020 was below expected numbers in February (five of seven journals) and March (four of seven) and above or equal to the expected numbers in April (six of seven), May (four of seven), and June (six of seven) (Figure 1) . Overall, only two journals experienced a decline (C: 6%, E: 129%) and five an increase (A: 13%, B: 11%, D: 9%, F: 2%, and G: 16%) in submissions over the COVID-19-affected months of 2020. A total of 68% of the possible 35 journal-month combinations (February-June 2020, 7 journals) had deviations above the expected number of submissions. Across all journals, we calculated that three more papers were submitted to these seven journals during these COVID-19-affected months than would have been expected based on the total observed submissions for the same period in other years, representing a 0.2% increase in submissions. There were fewer than expected submissions from North America (six of seven journals), Australia/Pacific (four of seven), and Asia (four of seven) during the February to June 2020 period ( Figure 2 ). The three other regions had more increases than decreases in submissions-Europe (four of seven journals), Africa Impact of COVID-19 on aquatic science community (four of seven), and South America (five of seven). Across all journals, there were 30 and 27 more submissions from Asia and Europe, respectively, while North America had 97 fewer submissions, than expected. The acceptance rate for undertaking reviews was lower in 2020 compared to the values prior to 2020 for three of six journals that provided these data, although rates were highly variable among journals ( Figure 3) . One of the journals had higher than usual acceptance rates (Journal F: þ16.7%) and one much lower (Journal A: -23.1%). Averaged across all six journals with these data, the review acceptance rate was only 1.8% lower for the COVID-19 period considered here. The average time manuscripts were in review could be calculated from data provided by five journals (Table 1) and is reported here annually, which was the most common time scale available ( Figure 4 ). All five of the journals experienced more rapid review times than expected, by between 4 (Journal G) and 7 days (Journal A). These absolute changes corresponded to between 4.6% and 21% faster than expected (Figure 4) . At an even finer temporal scale, Journal G provided data showing that the response time to accept an editors' review request (detrended) was faster by about half a day (0.67) for the 2020 period compared to the expected value of 3.08 days, a reduction of $21.7%. The worldwide disruption as a result of COVID-19 responses by national governments is unprecedented in the level of impact on society. Beginning in February 2020, measures imposed to slow the spread of the disease dramatically reduced domestic and international travel, shut workplaces and schools, and led to widespread job losses. This disruptive time has required a rethinking of values and work-private priorities at all levels of society (e.g. Corbera et al., 2020) and has been postulated to have an impact on scientific productivity (Ling, 2020; Myers et al., 2020) . We tested four hypotheses related to scientific productivity during the initial months of restrictions associated with COVID-19 (February-June 2020). Our analyses of aquatic scientist productivity were based on journal submissions and reviewing patterns, which are only one area of scientist productivity, but given the importance for career progression and funding success, an area that many scientists prioritize (e.g. Peoples et al., 2016) . It is also an activity that can be conducted even when travel or fieldwork is interrupted. Submission of manuscripts is a final stage in research, coming after a long series of steps, including grant submission, field, experimental or modelling research, and analysis and synthesis. Preparation of a manuscript may take weeks to months, and disruption in this final stage can often delay submission, resulting in fewer papers received by journals. If the shutdown period allowed scientists to instead focus on this final stage, we expected to see an increase in submission rates. Within this sample of seven journals, some had large declines in submissions and others had moderate increases. The variation among journals may reflect insufficient sample sizes, or differences between the researcher communities that submit to these journals. The monthly data showed that declines were greatest in the first months of the shutdown-February and March 2020, Figure 1 . Submission patterns per month in 2020 for seven major aquatic science journals, as a difference from the expected value (detrended linearly where there was an increase or decrease in submissions, as noted in Table 1) . with more typical rates from April onwards. This may reflect scientists settling into new routines. However, not all groups appear to be coping equally (e.g. Minello, 2020; Myers et al., 2020) . Overall, we observed no consistent decline and, therefore, cannot reject Hypothesis 1. This pattern varied by journal, which may also indicate differences in the scientific research areas that are most impacted (see Myers et al., 2020) . We contend that our sampled journals are representative of the science addressing freshwater, fisheries and marine ecosystem research questions, other marine science specialities such as physical oceanography are submitted to a different set of journals, and different patterns may be found for other research communities. We also cannot exclude the possibility that some scientists have been more productive and that this balanced the productivity decline in negatively impacted scientists. We note that we were unable to examine gender or other demographic differences (e.g. incountry restrictions, career stage, carer responsibilities, e.g. Myers et al., 2020) , as such information was not gathered by the journals included in this analysis. Such analyses may be attempted in future based on published articles for which ancillary information such as gender can be added via Internet searches. The geographic patterns in submission rate did not consistently show increases or decreases, which might be expected based on different applications of social isolation policies. Asia, which has had varied responses to COVID-19, showed a relative increase in submissions, as did Europe, where social distancing policies have been implemented less widely and/or commenced later in the year (June-July), while North America had an overall decrease. Thus, Hypothesis 2 cannot be rejected-at the scale we considered the patterns there was no consistent regional shift in submissions. The data for single-government institutions in Australia and the United States had larger decreases in productivity, close to 20%, and this is consistent with early responses to isolation and social distancing in the developed countries such as Australia and the United States. Australia's COVID-19 outbreak also followed severe societal disruption due to the worst bushfire season on record. It would be useful to examine submission rates from universities and other types of research agencies; however, such data are rarely gathered. The acceptance rate for reviews varied, with some journals showing relatively large declines, and others increases. Thus, Hypothesis 3 that COVID-19 would affect the willingness of scientists to review papers was not rejected. Interestingly, the time to review was quicker for all journals, thus rejecting Hypothesis 4. Scientists who did accept a request to provide a review were completing this task more quickly. This might be considered evidence for additional time allocated to reviewing, or fewer competing tasks. A single journal also provided data that showed the response to a request to review was received faster than in pre-COVID-19 years, indicating a greater willingness to review submissions by some scientists. Our results also provide comfort to editors-if more papers are going to be submitted during the COVID-19 months without an equivalent increase in reviewing, more pressure would be -25% -20% -15% -10% -5% 0% 5% Figure 3 . Reviewer invitation acceptance deviation from expected rates in 2020 for six of seven aquatic science journals (Journal D: no data were provided). Table 1 . "Total" represents the total deviation in submissions in 2020 for each geographic region across all seven journals. Impact of COVID-19 on aquatic science community placed on the existing reviewing community. However, we found no decline in submission and only a small decline in reviewing rate. From the data analysed here, we cannot determine whether the individuals who reduced their reviewing load also reduced their submissions, or if those who increased their number of submissions also increased their reviewing rate. It is well known that reviewing activity is not spread evenly among researchers (https:// publons.com/blog/spread-of-peer-review-workload/) and COVID-19 may exacerbate this in future. As with many facets of life under COVID-19, the experience so far in 2020 has offered a chance to reflect on effort allocated to writing manuscripts and reviewing tasks that add to the excessive workload already carried by many scientists. While these analyses cannot reveal individual narratives-e.g. some scientists have had higher productivity and others lower-we have shown that the COVID-19 disruption has not dramatically reduced productivity by the aquatic science community as indicated by total manuscript submissions or review acceptance rates to seven journals, and from two large national research institutes, that we assume are representatives of the overall aquatic science community. In the coming years, it will be interesting to track the long-term impact on publishing and other forms of scientific productivity, particularly to quantify this disruption on the career progression for COVID-19-disadvantaged individuals and groups. Supplementary material is available at the ICESJMS online version of the manuscript. . The time in review deviation from expected in 2020 for four of seven aquatic science journals. For example, Journal A reviews in 2020 were completed 7 days faster than expected, a time reduction of 21%. (Journals C, D, and F: data were not provided or available or were unsuitable). Learning from single extreme events Extreme events defined-a conceptual discussion applying a complex systems approach Academia in the time of COVID-19: towards an ethics of care Shut-in scientists are spending more time on research papers This pandemic is not an extended sabbatical In the wake of COVID-19, academia needs new solutions to ensure gender equity The pandemic and the female academic Unequal effects of the COVID-19 pandemic on scientists How early-career scientists are coping with COVID-19 challenges and fears Twitter predicts citation rates of ecological research On detecting ecological impacts of extreme climate events and why it matters The decline of women's research production during the coronavirus pandemic Scientist Mothers Face Extra Challenges in the Face of COVID-19 Submission and reviewing data are considered proprietary information by journals, and we are grateful for the access we were granted. We appreciate assistance in obtaining approval to use journal data from Max Finlayson, George Rose, Marisa Spiniello, Yong Chen, K. Dave Hambright, Christine Paetzold, Lia Curtin, Emily Davies, and Georgina Smith. Data on institutional submissions were provided by Meryn Scott (CSIRO library) and Laurie Barak (NOAA SWFSC). Assistance in data preparation was expertly provided by Vicki Walters, as was review by Brett Moloney and Rodrigo Bustamante.