Evidence Summary

 

Manuscripts Published in a Specific Chemistry Journal Must Be Both Important and Suitable According to Peer Reviewers

 

A Review of:

Bornmann, L., & Daniel, H.-D. (2010). The manuscript reviewing process: empirical research on review requests, review sequences, and decision rules in peer review. Library & Information Science Research, 32(1), 5-12. https://doi.org/10.1016/j.lisr.2009.07.010

 

Reviewed by:

Michelle DuBroy

Discipline Librarian, Researcher Services

Griffith University

Southport, Queensland, Australia

 

Received: 25 Feb. 2021                                                              Accepted:  9 Apr. 2021

 

 

cc-ca_logo_xl 2021 DuBroy. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

 

 

DOI: 10.18438/eblip29936

 

 

Abstract

 

Objective – To examine the peer review process at a single journal.

 

Design – Analysis of business records.

 

Setting – Peer review system of a single journal.

 

Subjects – Documents produced when reviewing manuscripts submitted for publication to journal Angewandte Chemie International Edition and reviewed in the year 2000.

 

Methods – Peer review process information was extracted from the journal’s archives. Various aspects, such as review sequences and decision rules, were analysed and summarised in tables.

 

Main results – Of the 1899 manuscripts reviewed in the year 2000, 46% (n = 878) were accepted for publication and 54% (n = 1021) were rejected. On average, a manuscript received 2.6 reviews before an editor made a publication decision. Just over half (n = 962, approx. 51%) of manuscripts were subject to two review steps. A small number of manuscripts (n = 104, approx. 5.5%) were subject to 5, 6 or 7 review steps. The more steps an article was subject to, the greater likelihood it would be accepted. Editors “generally follow a so-called clear-cut rule” (p.11) in which manuscripts accepted for publication must be considered both important and suitable for publication by at least two peer reviewers.

 

Conclusion – The results “give a sense of commitment [and care] ...probably typical of most prestigious journals” (p.11).

 

Commentary

 

Peer review is a fundamental part of the research process. Despite its importance, traditional peer review is said to lack sufficient transparency, accountability, and consistency (Ross-Hellauer, 2017).

 

This study was likely the first to examine the inner workings of the peer review system. Years later, our understanding of peer review has greatly increased, yet many challenges remain (Tennant, 2018).

 

The study was evaluated using a critical appraisal tool (Perryman & Rathbun-Grubb, 2014). While the study was successful in opening the “black box” (p.5) of peer review at a single journal, it also suffered from a few weaknesses.

 

A concise literature review gives adequate background and context. The methods generally appear logical for addressing the stated objectives. Further, the authors reported their findings thoroughly and made good use of tables.

 

Reproducibility is unfortunately limited. Many details regarding the research process are unknown. Did the authors code the information in some way? Did they use software for the analysis? Readers can only guess.

 

Further, the intended audience and utility of this study is unclear. The authors refer to library “collection managers” (p. 11), but they do not suggest how this group might apply the results of the study to their practice. Additionally, the authors do not discuss how other users, such as researchers, might make practical use of the study’s findings. Clearer research questions, focused on supplying readers with meaningful answers (Doolan & Froelicher, 2009), may have given the analysis needed direction.

 

Overall, this notable study revealed details regarding an important, yet relatively opaque, part of the research ecosystem. It may be of some interest to academic librarians who support researchers in their scholarly publishing. However, clearer, more audience-focused research questions may have helped increase the study’s usefulness. Additionally, more transparent research processes would have enabled other researchers to productively build on their work.

 

References

 

Doolan, D. M., & Froelicher, E. S. (2009). Using an existing data set to answer new research questions: a methodological review. Research and Theory for Nursing Practice, 23(3), 203-215. https://doi.org/10.1891/1541-6577.23.3.203

 

Perryman, C., & Rathbun-Grubb, S. (2014). The CAT: A generic critical appraisal tool. https://www.jotform.us/cp1757/TheCat

 

Ross-Hellauer, T. (2017). What is open peer review? A systematic review [version 2; peer review: 4 approved]. F1000Research, 6(588). https://doi.org/10.12688/f1000research.11369.2

 

Tennant, J. P. (2018). The state of the art in peer review. FEMS Microbiology Letters, 365(19). https://doi.org/10.1093/femsle/fny204