Evidence Summary

 

Open Access Works are as Reliable as Other Publishing Models at Retracting Flawed Articles from the Biomedical Literature

 

A Review of:

Peterson, G.M. (2013). Characteristics of retracted open access biomedical literature: a bibliographic analysis. Journal of the American Society for Information Science and Technology. 64(12), 2428-2436. doi: 10.1002/asi.22944

 

Reviewed by:

Elizabeth Stovold
Information Specialist, Cochrane Airways Group
St George’s, University of London
Tooting, London, United Kingdom
Email: estovold@sgul.ac.uk

 

Received: 16 Jun. 2014   Accepted: 23 Jul. 2014

 

 

cc-ca_logo_xl 2014 Stovold. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

 

Abstract

 

Objective – To investigate whether the rate of retracted articles and citation rates post-retraction in the biomedical literature are comparable across open access, free-to-access, or pay-to-access journals.

Design – Citation analysis.

Setting – Biomedical literature.

Subjects – 160 retracted papers published between 1st January 2001 and 31st December 2010.

Methods – For the retracted papers, 100 records were retrieved from the PubMed database and 100 records from the PubMed Central (PMC) open access subset. Records were selected at random, based on the PubMed identifier. Each article was assigned a number based on its accessibility using the specific criteria. Articles published in the PMC open access subset were assigned a 2; articles retrieved from PubMed that were freely accessible, but did not meet the criteria for open access were assigned a 1; and articles retrieved through PubMed which were pay-to-access were assigned a 0. This allowed articles to be grouped and compared by accessibility.

Citation information was collected primarily from the Science Citation Index. Articles for which no citation information was available, and those with a lifetime citation of 0 (or 1 where the citation came from the retraction statement) were excluded, leaving 160 articles for analysis. Information on the impact factor of the journals was retrieved and the analysis was performed twice; first with the entire set, and second after excluding articles published in journals with an impact factor of 10 or above (14% of the total). The average number of citations per month was used to compare citation rates, and the percentage change in citation rate pre- and post-retraction was calculated. Information was also collected on the time between the date the original article was published and the date of retraction, and the availability of information on the reason for the retraction. 

Main results – The overall rate of retracted articles in the PMC open access subset compared with the wider PubMed dataset was similar (0.049% and 0.028% respectively). In the group with an accessibility rating of 0, the change in citation rate pre- and post-retraction was -41%. For the group with an accessibility rating of 1, the change was -47% and in those with a rating of 2, the change in citation rate was -59%. Removing articles published in high impact factor journals did not change the results significantly. Retractions were issued more slowly for free access papers compared with open or fee-based articles.  The bibliographic records for open access articles disclosed details of the reason for the retraction more frequently than free, non-open papers (91% compared to 53%).

Conclusion – Open access literature is similar in its rate of retraction and the reduction in post-retraction citations to the rest of the biomedical literature, and is actually more reliable at reporting the reason for the retraction.

 

Commentary

 

Open access is a growing movement, and therefore an examination of the quality of biomedical literature published through an open access arrangement is of great importance. Retracted articles in the biomedical literature are relatively rare, but can have a big impact if the error is of such significance that it could negatively affect research or practice based on the evidence presented in the retracted article. To illustrate this the author cites the example of a discredited paper on MMR vaccines which resulted in a reduction of the uptake of MMR vaccines in the UK.

 

This study was evaluated using the critical appraisal tool developed by Perryman (2009). The inclusion and exclusion criteria of the articles were appropriate, and reasons for excluding articles from the analysis were explicitly stated. The sources used to retrieve retracted articles and citation information were clearly reported and the choice justified. The author used the number of average citations per month to compare citation rates which controlled for differences in the length of the pre- and post-retraction periods and allowed articles published at different times during the decade under study to be compared. The author also examined other factors which may affect the citation rates following retraction, such as the impact factor of the journal. The author discussed limitations of the study; it was noted that there was no control group for the study, the sample of retracted articles were retrieved from a single source (PubMed), and it may not be possible to extrapolate these findings to subject areas outside of biomedicine, or to other types of post-publication changes in the literature.

               

Potential limitations not discussed in the paper were that a single author conducted the study, therefore judgements on the accessibility value of each paper were not validated, and the sample of retracted articles in the analysis was small. The study by Furman, Jensen & Murray (2012) used a sample of 677 retracted articles and 1,340 control articles, compared to 160 in this study. However, the sample size in this study is restricted by the availability of retracted articles published under an open access arrangement.

 

Overall this study presents a thorough evaluation of the characteristics of retracted articles compared by their accessibility and finds that open access journals are as reliable as the traditional publishing models in retracting flawed work. This is reassuring for librarians and searchers who may be recommending open access journals to researchers and practitioners in the biomedical field as sources of reliable evidence.

 

 

References

 

Furman, J. L., Jensen, K., & Murray, F. (2012). Governing knowledge in the scientific community: Exploring the role of retractions in biomedicine. Research Policy, 41(2), 276–290. doi: 10.1016/j.respol.2011.11.001

 

Perryman, C. (2009). Critical appraisal tool for bibliometric studies. In Carol Perryman. Retrieved from http://cperryman.com/Critical%20appraisal%20tool%20-%20Bibliometric%20study.doc