Citations to Wikipedia in Chemistry Journals: A Preliminary Study

Bradley Brazzeal
Engineering & Forest Resources Reference Librarian
Mississippi State University Libraries
Mississippi State, Mississippi
bbrazzeal@library.msstate.edu

Abstract

Wikipedia has been the subject of an increasing number of studies. Many of these have focused on the quality of Wikipedia articles and the use of Wikipedia by students. Little research has focused on the use of Wikipedia by scholars. This study helps to fill that gap by examining citations to Wikipedia in chemistry journals from three major publishers over a five year period. The study reports the number of citations to Wikipedia and describes how Wikipedia is being cited. The results show that, while only a small percentage of all articles contained a citation to Wikipedia, it is in fact being cited as a credible information source in articles in major chemistry journals.

Introduction

Founded in 2001, Wikipedia contains over 17 million articles covering a wide array of topics. It was created to be a "free encyclopedia" rather than an outlet for original research. In most cases, articles can be edited by any registered user, and if an article does not currently exist, then registered users can write one themselves. The administrators of Wikipedia acknowledge that allowing anyone to contribute is both a strength and a weakness of the site, and they provide words of caution regarding the use of Wikipedia as a source for research papers (Wikipedia: About [Updated 2011 Mar 12]). In spite of this weakness, Wikipedia has made inroads into the information landscape of various disciplines, including chemistry.

Links to Wikipedia articles have been integrated into a number of scientific information sources, including Science.gov, Common Chemistry, and ChemSpider. Science.gov, an initiative of the United States government, serves as "a gateway to government science information and research results" (Science.gov: About [Updated 2011 Mar 16]), while Common Chemistry, a project of the American Chemical Society, provides "CAS Registry Number, chemical names (both formal and common), molecular formulas, and structures or sequences for ~7,900 chemicals of widespread general public interest" (About Common Chemistry [Updated 2010 Oct]).Similarly, ChemSpider, a project of the Royal Society of Chemistry, "aggregates chemical structures and their associated information into a single searchable repository and makes it available to everybody" (What is ChemSpider? [Cited 2011 May 5]). The integration of links to Wikipedia articles in these scientific resources indicates that the responsible institutions believe that Wikipedia has a place in the world of scientific information.

The purpose of the present study is to see the extent to which scholars cite Wikipedia articles in scholarly journals. The author wanted to avoid articles that might focus on pedagogical, social, or technical aspects of Wikipedia, and chose to focus on scholarly chemistry journals. The importance of examining actual citations of chemists rather than only surveying their opinions lies in the fact that the decision to cite Wikipedia as an information source in scholarly articles is one of the greatest endorsements that it can receive (Okoli 2009).

Literature Review

Wikipedia is the subject of an increasing number of scholarly studies. A search on Scopus for articles or reviews with the term "Wikipedia" in the article title, abstract, or keywords yielded 452 results on March 16, 2011. Okoli (2009) identified four categories of articles that cite Wikipedia: (1) how and why Wikipedia works, (2) the reliability of Wikipedia, (3) Wikipedia as a data source, and (4) applications of Wikipedia in education and economics. It is the third category, which could be expanded to "Wikipedia as an information source," that the present study investigates, focusing specifically on the discipline of chemistry.

Some of the benefits and drawbacks of using Wikipedia for chemistry-related information are presented in the following excerpts of articles and commentaries in scholarly journals:

"Wikipedia has emerged as a common tool that many use regularly for at least an initial attempt to find information about a new subject ... Some types of vandalism can be detected automatically ..." (Martinsen 2006, 196)

"Wikipedia ... is useful as a popular information resource. One example is the article on Aspirin ... which displays the chemical structure diagram ... and provides fairly extensive descriptions of its history, trademark issues, synthesis, therapeutic uses, adverse effects, and more, and includes an extensive list of literature references" (Roth 2008, 232).

"Although relatively few chemists contribute to Wikipedia, the quality of its chemical content is high and increasing" (Murray-Rust 2008, 649).

"The chemistry articles are not without issue and the drug and chemboxes, specifically, have been shown to contain errors. However, the advantage of a wiki is that changes can be made within a few keystrokes and the quality is immediately enhanced" (Williams 2008, 498).

"As with all open source projects, the final product emerges gradually through a large number of iterative changes. In amongst reports of the thousands of spurious edits of the page for US presidents, it is worthwhile remembering that Wikipedia currently contains over five million articles, in 250 languages (1.5 million in English), has emerged within the last six years and is available free of charge" (Todd 2007).

The most comprehensive examination of Wikipedia chemistry resources is provided by Walker (2010). Walker notes that there are seven "WikiProjects" that try to monitor chemistry-related articles, and he points out that efforts such as ChemSpider and Common Chemistry are also helping to improve Wikipedia articles. While enthusiastic about the possibilities of Wikipedia, Walker does caution that "... if the information is to be published, other sources should be used to verify the accuracy" (p. 87). Concerns about citing a Wikipedia article that may change in the future can also be addressed by using a Wikipedia generated link to the version used (Walker 2010).

Scholars cite information sources for a number of reasons. As Rubens (2001) indicates, "Properly citing sources satisfies legal requirements, strengthens the author's argument, maintains intellectual integrity, and provides a valuable resource for the reader" (p. 195). While citing is an essential aspect of scholarly communication, the choice of what to use for sources may be influenced by the information-seeking behavior of authors. Connaway & Dickey (2010) presented a meta-analysis of 12 user studies conducted by OCLC, Research Information Network, and JISC. These studies examined the information-seeking behavior of both students and researchers, and among the common themes found was "the increasing centrality of Google" (p. 27). Niu et al. (2010) found that just over half of science and engineering researchers at five research universities in the United States, start their "search process" on Google (p. 876). This reliance on Google could contribute to researchers' use of Wikipedia, since Wikipedia is often among the first results of Google and other popular search engines for scientific topics (Laurent & Vickers 2009; Page 2010). The use of Google and Wikipedia could be illustrative of the observation by Prabha et al. (2007) that "[t]he very abundance of information makes it crucial for information seekers to decide what information is enough to meet their objectives" (p. 77).

While citations to Wikipedia in the legal literature have been well studied (e.g. Baker 2009), few systematic studies of citations to Wikipedia have been conducted for the literature of other fields. One brief study on the topic appeared in an editorial for the journal Webology by Noruzi (2009), who reported the results of a cited reference search in the Web of Science for articles in which Wikipedia was listed as the cited work. The search yielded 263 results for the period 2004 to 2009 and demonstrated the wide range of journal subject areas that cite Wikipedia, but it was limited by the way the Web of Science tracks such citations. Another study appeared in a blog post by Spiro (2008), who searched for the term "Wikipedia" in the Project Muse and JSTOR databases. There were 167 results from 2002 to 2008, with an increase for nearly every year. Spiro also provided various categorizations of the citations. More recently, Dooley (2010) examined 250 results from a search for "Wikipedia" in the Academic OneFile database and found that 249 of those had Wikipedia as "a source of scholarly information."

This current study focuses on citations to Wikipedia in chemistry journals from three major publishers and examines not only the number of citations to Wikipedia but also how Wikipedia is cited.

Methods

In order to examine citations to Wikipedia in scholarly chemistry journals, the author selected three publishers that (1) publish a substantial number of chemistry journals and (2) have web sites that can be effectively searched to see if the term "Wikipedia" occurs anywhere in an article, including inside a URL. Selected publishers were the American Chemical Society (ACS), Elsevier, and Springer. Another major publisher of chemistry journals, the Royal Society of Chemistry, was excluded because the web site could not be effectively searched in the manner described above. Items that do not typically undergo peer review, such as editorials, letters to the editor, and magazine articles, were excluded from the study.

The journal portal for each publisher was searched, and a Microsoft Access database was created with bibliographic information for articles citing Wikipedia. Each article was then examined to identify all in-text and reference list citations to Wikipedia. Those references were added to the database, as were the number of times an article cited Wikipedia and how Wikipedia was cited in the article. The latter information was important because the author was especially interested in determining if Wikipedia was being cited for information such as chemical and physical properties of substances, such as one would find in the CRC Handbook of Chemistry and Physics and other standard reference sources. Any reference to Wikipedia as an unreliable source of information was not evaluated. The following categories of citations were developed to do this.

General Scientific Information. This category included citations used to support statements about topics directly related to science, technology, engineering, and mathematics. This included such information as properties of chemicals, mathematical and medical information, etc. Two subcategories of General Scientific Information were also developed.

The Mathematical Equations subcategory included citations to Wikipedia to document a particular mathematical formula that was given in the article.

The Numerical Values of Physical & Chemical Properties subcategory included citations to Wikipedia that documented physical and chemical properties (such as density, melting point, etc.) that were expressed numerically.

General statements of mathematical concepts and non-numerical descriptions of these items were only included in the broader category of General Scientific Information.

General Non-Scientific Information. This category was used for those citations to information dealing with such things as historical events, biographical information of scientists, and policy and business information. Also included were references that merely spoke about the general uses of an element, compound, etc. For simplification, if a citation in the General Scientific Information category also included General Non-Scientific Information, then it was only counted as General Scientific Information.

Further Scientific Information and Further Non-Scientific Information. Sometimes, authors did not use Wikipedia to document what was written in the article but rather as a recommended source for the reader to find additional information. For simplification, if a citation in the General Scientific Information category also included one of the further information categories, then it was only counted as General Scientific Information.

Image Credits. This category included instances where Wikipedia was cited as a source for images used in an article, either in the original form or as an adaptation.

Results and Discussion

The first citations to Wikipedia appeared in 2005, and from 2005 to 2009, there were a total of 370 research articles with such citations. These articles appeared in 147 of the 312 chemistry journals published by the three publishers during this period; just under half (47%) of all of the journals had at least one article that cited Wikipedia. Because most chemistry journals publish hundreds of articles in a given year, over the five years only a very small percentage of all articles by the publishers contained a citation to Wikipedia. For example, the journals listed in the Multidisciplinary Chemistry category of the 2009 Journal Citation Reports (JCR) had an average of 268 articles each in that year.

Among the most popular journal evaluation metrics is JCR's Impact Factor. Seventy-seven percent of all the journals that cited Wikipedia were ranked in the first or second quartile by JCR, suggesting that these are among the high quality journals in the field. This is in line with the research of Baker (2009), who found citations to Wikipedia in 29 of the top 30 law reviews, according to a listing by Washington & Lee University Law School. The selection criteria for journals in JCR can be found at Thomson Reuters Journal Selection Process ([Cited 2011 May 5]).

There were only six articles with citations in 2005; this rose to 31 the following year. The number of articles then shot up to 84 in 2007, 86 in 2008, and 163 in 2009. Citations to multiple entries were found in 44 articles (12%). Fifty four articles (15%) used only in-text citations for Wikipedia instead of including Wikipedia in the reference list. The Multidisciplinary Chemistry journals in the 2009 JCR showed an average of 38 references per article. If that is taken as representative of most chemistry journals, then it is clear that authors of the articles in this study were not relying heavily on Wikipedia. Yet, it is significant that Wikipedia was being cited at all.

Table 1 provides a summary of the number and percentage of articles that contained citations to Wikipedia in the various categories. The largest category of citations was General Scientific Information, which was found in 63% of the 370 articles. Within this category, there were 10 articles (3% of the total number of articles) that cited Wikipedia for specific mathematical equations and 33 articles (9% of the total number of articles) that cited Wikipedia for Numerical Values of Physical & Chemical Properties. This confirms that there are scientists who view at least some of the scientific information on Wikipedia as sufficiently reliable.

Category	Number of Articles with Citations in Category	Percentage of Total Articles
General Scientific Information	233	63
-- Mathematical Equations	10*	3*
-- Numerical Values of Physical & Chemical Properties	33*	9*

General Non-Scientific Information	67	18

Further Scientific Information	40	11

Further Non-Scientific Information	7	2

Image Credits	15	4

Undetermined	16	4

Total:	378	102

Table 1. Classification of citations to Wikipedia. Total is greater than the number of articles because some articles cited multiple Wikipedia entries in ways that fell into more than one category. (* Numbers and percentages for the two subcategories are not included in the totals.)

Conclusion

This study augments the body of research into scholars' use of Wikipedia by examining actual citations to Wikipedia in scholarly chemistry journals. While the number of articles with such citations is small, it is significant, given the controversies about Wikipedia outlined and addressed by Walker (2010), that the scholarly articles included the citations, and the editors and reviewers of these journals allowed them. Since none of the citations in this study were negative citations, their presence in the articles could be interpreted as an endorsement of Wikipedia (Okoli 2009). The majority of these citations were used for information of a scientific nature, and these included some citations for mathematical equations as well as numerical values of physical and chemical properties found in common reference works. Baker (2009) suggested that the legal profession should "discourage most citations to Wikipedia" (p. 42). This study of chemistry journals is descriptive in nature and does not judge the appropriateness of citing Wikipedia, but if the number of citations to Wikipedia continues to increase, the chemistry discipline may need to have a collective discussion about the value and acceptability of the resource.

A limitation of this study is that it could not measure the motives for citing Wikipedia. For example, did citations for scientific information such as physical and chemical properties reflect the authors' support of "open information" that has been promoted by Murray-Rust (2008), or do the citations reflect the increased dependence on Google (Connaway & Dickey 2010; Niu et al. 2010), which often puts Wikipedia among the first results of a search (Laurent & Vickers 2009; Page 2010)? Such information could be gathered in future studies that use techniques such as surveys and focus groups. Future research could also use the method outlined in this article to examine citations to Wikipedia in journals in other disciplines or to determine if citations to Wikipedia in chemistry journals continue to show an upward trend in the coming years.

Acknowledgement

The author would like to thank Rachel Cannady, David Nolen, two anonymous reviewers, and the Refereed Articles Editor for their valuable comments on drafts of this article.

References

About Common Chemistry. [Internet]. [Updated 2010 October]. Columbus (OH): Chemical Abstracts Service. Available from: http://www.commonchemistry.org/about.aspx

Baker, D. 2009. Citations to Wikipedia in law reviews. University of Houston Law Center No. 2009-A-42. [Internet]. [Cited 2011 Jun 24]. Available from: http://ssrn.com/abstract=1525619

Connaway, L.S. & Dickey. T.J. 2010. The Digital Information Seeker: Report of the Findings from Selected OCLC, RIN, and JISC User Behaviour Projects. [Bristol (UK)]: Higher Education Funding Council for England.

Dooley, P. 2010. Wikipedia and the two-faced professoriate. Proceedings of the 6th International Symposium on Wikis and Open Collaboration, 2010 Jul 7-9; Gdansk, Poland. New York: ACM.

Laurent, M.R. & Vickers, T.J. 2009. Seeking health information online: Does Wikipedia matter? Journal of the American Medical Informatics Association 16(4): 471-479.

Martinsen, D.P. 2006. Technology transfer: From society to the lab and classroom. ACS Chemical Biology 1(4): 195-197.

Murray-Rust, P. 2008. Chemistry for everyone. Nature 451(7 Feb): 648-651.

Niu, X., Hemminger, B.M., Lown, C., Adams, S., Brown, C. Level, A., McLure, M., Powers, A., Tennant, M.R., & Cataldo, T. 2010. National study of information seeking behavior of academic researchers in the United States. Journal of the American Society for Information Science and Technology 61(5): 869-890.

Noruzi, A. 2009. Wikipedia popularity from a citation analysis point of view. Webology [Internet]. [Cited 2009 Dec 10]; 6(2). Available from: http://www.webology.org/2009/v6n2/editorial20.html

Okoli, C. 2009. A brief review of studies of Wikipedia in peer-reviewed journals. Proceedings of the Third International Conference on Digital Society, 2009 Feb 1-6; Cancun, Mexico. Los Alamitos (CA): IEEE Computer Society, p. 155-160.

Page, R.D.M. 2010. Wikipedia as an encyclopedia of life. Organisms Diversity & Evolution 10(4): 343-349.

Prabha, C., Connaway, L.S., Olszewski, L. & Jenkins, L.R. 2007. What is enough? Satisficing information needs. Journal of Documentation 63(1): 74-89.

Roth, D.L. 2008. Web-accessible chemical compound information. Journal of Electronic Resources in Medical Libraries 5(3): 228-242.

Rubens, P. 2001. Science & Technical Writing, 2nd ed. New York: Routledge.

Science.gov: About. [Internet]. [Updated 2011 Mar 16]. Oak Ridge (TN): Office of Scientific and Technical Information (US). Available from: http://www.science.gov/about.html

Spiro, L. 2008. Is Wikipedia Becoming a Respectable Academic Source? Digital Scholarship in the Humanities [Internet]. [Updated 2008 Sep 1]. Available from: http://digitalscholarship.wordpress.com/2008/09/01/is-wikipedia-becoming-a-respectable-academic-source/

Thomson Reuters Journal Selection Process. [Internet]. [Cited 2011 May 5]. New York (NY): Thomson Reuters. Available from: http://thomsonreuters.com/products_services/science/free/essays/journal_selection_process/

Todd, M.H. 2007. Open access and open source in chemistry. Chemistry Central Journal [Internet]. [Cited 2010 Dec 8]; 1. Available from: http://www.journal.chemistrycentral.com/content/1/1/3

Walker, M.A. 2010. Wikipedia as a resource for chemistry. In: Belford, R.E., Moore, J.W. & Pence, H.E., editors. Enhancing Learning with Online Resources, Social Networking, and Digital Libraries. Washington (DC): American Chemical Society. p. 79-92.

What is ChemSpider? [Internet]. [Cited 2011 May 5]. Cambridge (UK): Royal Society of Chemistry. Available from: http://www.chemspider.com/About.aspx

Wikipedia: About. [Internet]. [Updated 2011 Mar 12]. San Francisco (CA): Wikimedia Foundation. Available from: http://en.wikipedia.org/wiki/Wikipedia:About

Williams, A.J. 2008. A perspective of publicly accessible/open-access chemistry databases. Drug Discovery Today 13(11-12): 495-501.

Previous	Contents		Next
Issues in Science and Technology Librarianship		Fall 2011
DOI:10.5062/F4057CV7