Previous Contents Next
Issues in Science and Technology Librarianship
Summer 2008
DOI:10.5062/F4FF3Q9G

[Board accepted]

Comparison of Journal Citation Reports and Scopus Impact Factors for Ecology and Environmental Sciences Journals

Edward Gray
Public Services Librarian
teddy.gray@duke.edu

Sarah Z. Hodkinson
Library Assistant
sarah.hodkinson@duke.edu

Biological and Environmental Sciences Library
Duke University
Durham, North Carolina

Copyright 2008, Edward Gray and Sarah Z. Hodkinson. Used with permission.

Abstract

Impact factors for journals listed under the subject categories "ecology" and "environmental sciences" in the Journal Citation Reports database were calculated using citation data from the Scopus database. The journals were then ranked by their Scopus impact factor and compared to the ranked lists of the same journals derived from Journal Citations Reports. Although several titles varied significantly in impact factor and rank, the Journal Citation Reports and Scopus lists had a high degree of statistical similarity.

Introduction

The Thomson Scientific impact factor is a popular tool that is used to measure the influence of scholarly journals. Nearly fifty years ago, Eugene Garfield developed this method as a way to use the Thomson Scientific (formerly Institute for Scientific Information, ISI) citation indexes to analyze the frequency with which the average journal article is cited (Garfield & Sher 1963; Garfield 1972). The impact factor of a journal is calculated by dividing the references cited in one year by the number of citable articles published in the same journal over the previous two years. This ratio is published annually in Thomson Scientific's Journal Citation Reports (JCR), along with a number of other quantitative tools for comparing and evaluating scholarly journals. For years, the impact factor has been used to "clarify the significance of absolute citation frequencies" (Thomson Scientific 1994), and as a result its applications have gained great significance in the world of academia (Monastersky 2005). Academic librarians frequently use impact factors to help them decide which journals are important enough to subscribe to and which subscriptions may be canceled (Barschall 1988; Coleman 2007).

While impact factor data were originally used extensively in journal marketing to rank and compare scholarly journals for prospective subscribers, it has become a way of ranking the scientists who publish articles in these journals. Since the importance of scientific research is notoriously difficult to evaluate quantitatively, many universities now make funding and tenure decisions based on the average impact factor value, or prestige, of the journals that a scientist has published in. This may seem to be a reliable tool to assess the importance of a particular scientist's research, but there have been numerous reports that show the dangers of overestimating the value of the impact factor (Seglen 1997; Opthof 1997; Coleman 1999). The impact factor of a journal may be artificially inflated when it frequently includes review articles and letters, and there is also a bias against articles that are not published in English. Journals that publish articles on botanical or zoological taxonomy may also have lower impact factors because they often rely on older references, and references to taxonomic names are not included in the references list at the ends of these papers (Werner 2006).

Since the widely used JCR impact factors are all based on data from journals indexed in Web of Science, the researchers of the present study were interested in comparing the impact factors calculated from citation data from another database. Elsevier's Scopus, which became available in 2004, has been considered a competitor to Web of Science because of its citation tracking capabilities (Bakkalbasi et al. 2006). Scopus does not rank journals according to "impact factor," using instead a measure called the h-index that was developed in 2005 by Hirsch to evaluate the impact of journals. However, the data required to calculate an impact factor using Garfield's ratio are all available in the Scopus database (Pislyakov 2007). If impact factor is the most widely used tool for evaluating the importance of a journal and thus the worth of its writers, it would be wise to determine whether or not calculated impact factors vary significantly between JCR and Scopus.

Methods

Impact factors were obtained for the 116 ecology titles and the 160 environmental sciences titles listed in the 2007 edition of JCR. In this latest edition of JCR, cites to recent articles (the numerator of the impact factor equation) were from 2007, while citable articles (the denominator) were from 2005 and 2006. JCR does not state what constitutes a citable article. Generally, the number of citable articles in JCR corresponds to the number of research and review articles indexed in Web of Science for any particular journal, however, the two numbers don't always correspond.

Impact factors were generated in Scopus by individually searching in the source title field each journal from the JCR ecology and environmental sciences lists. Dates were limited to 2005 through 2006. If Scopus didn't index a journal from the JCR list during this time period, the title was eliminated from further consideration. Once results were obtained for each journal, all the citations were selected and the citation tracker feature was used to determine the number of times these articles were cited in 2007. These numbers were placed in an Excel spreadsheet and were divided against the number of citable articles from JCR to come up with Scopus impact factors.

Using the rank formula in Excel, the journals were ranked by both their JCR Scopus impact factors. For each journal, the Scopus rank was subtracted from the JCR rank to determine a change in rank. Finally, the rankings were entered into an online statistical application to determine their Spearman rank-order correlation coefficients, which determine statistical similarity between two lists (Lowry 2008).

Results and Discussion

Tables 1 and 2 show the JCR and Scopus impact factors, the JCR and Scopus ranks, and the change in rank from JCR to Scopus for the top 50 ecology and environmental sciences titles, respectively. Although several of the titles changed greatly in rank (e.g., Wildlife Monographs dropped 38 places from 17 in JCR to 55 in Scopus for the ecology list), the JCR and Scopus ranks had a high degree of statistical similarity. The Spearman rank-order correlation coefficient was .9818 for the ecology titles and .9823 for the environmental science titles.

Several journals warrant closer examination because of their changes in impact factor and rank. Although Bulletin of the American Museum of Natural History only dropped one spot from number one in JCR ecology rank to number two in Scopus ecology rank, its Scopus impact factor (16.385) was nearly 4.7 points less than its JCR impact factor (11.692). Several factors could account for this steep drop in impact factor. JCR listed 13 citable articles that determined its impact factor. Scopus, which did not match Web of Science in its indexing of this title, only returned 11 articles for this journal. Because we relied on the number of citable articles from JCR to create our Scopus impact factor, any title that isn't completely indexed by Scopus will have an underinflated impact factor. Had we used the 11 articles indexed in Scopus as our number of citable articles, the impact factor would have increased by over two points to 13.818.

The high JCR impact factor for Bulletin of the American Museum of Natural History might also result from the data in JCR not matching the data in Web of Science. When we performed a publication search in Web of Science for Bulletin of the American Museum of Natural History for 2005 and 2006 and selected the "create citation report" feature, the database showed that the journal received only 153 cites in 2007; JCR (which is supposedly using Web of Science data) returned 213 cites. Using 153 cites instead of 213 cites creates a JCR impact factor of 11.769, which is more in line with the Scopus number. In case JCR is using incorrect citations in Web of Science to come up with a cite count, we also performed a search for the journal in the cited reference search. Articles from 2005 and 2006 were cited 203 times in 2007. If we use 203 cites, we get a JCR impact factor of 15.615. Because JCR doesn't indicate what cites are used in their data, we can't be certain if the discrepancy between the JCR and Scopus impacts is the result of incomplete indexing by Scopus, poor data in JCR, our methodology of relying on JCR for the number of citable articles, or some combination of all three.

Wildlife Monographs was another journal that shows a major change between the two different sets of impact factors as that title dropped from 17th in the JCR rank to 55th in the Scopus rank for ecology titles. Both Web of Science and Scopus indexed 10 articles for the journal from 2005 to 2006, so incomplete indexing by Scopus can't be the cause of the difference. Using the "create citation report" feature in Web of Science, we got only 10 cites for Wildlife Monographs compared to the 39 listed in JCR. Using the cited reference search, we got 33 cites for the journal. If we rely on the Web of Science numbers instead of the JCR numbers, the impact factor for Wildlife Monographs is either 1.000 or 3.300, which places the Scopus impact factor of 1.700 nearly in the middle.

Table 1: Impact Factors, Ranks, and Change in Ranks for the Top 50 Ecology Journals

Journal Title

JCR Impact Factor

Scopus Impact Factor

JCR Rank

Scopus Rank

Change in Rank

Bulletin of the American Museum of Natural History

16.385

11.692

1

2

-1

Trends in Ecology & Evolution

14.797

14.993

2

1

1

Annual Review of Ecology Evolution and Systematics

10.340

10.440

3

3

0

Ecology Letters

8.204

8.292

4

4

0

Ecological Monographs

8.117

8.117

5

5

0

Molecular Ecology

5.169

5.151

6

6

0

Ecology

4.822

4.775

7

8

-1

Global Change Biology

4.786

5.000

8

7

1

American Naturalist

4.543

4.457

9

9

0

Evolution

4.502

4.441

10

10

0

Global Ecology and Biogeography

4.435

4.259

11

12

-1

Journal of Ecology

4.422

4.422

12

11

1

Frontiers in Ecology and the Environment

4.269

4.241

13

13

0

Journal of Applied Ecology

4.220

4.174

14

14

0

Conservation Biology

3.934

3.926

15

15

0

Journal of Evolutionary Biology

3.920

3.911

16

16

0

Wildlife Monographs

3.900

1.700

17

55

-38

Journal of Animal Ecology

3.747

3.720

18

17

1

Ecological Applications

3.571

3.562

19

18

1

Journal of Biogeography

3.539

3.503

20

19

1

Biological Conservation

3.296

3.354

21

20

1

Paleobiology

3.225

3.270

22

21

1

Functional Ecology

3.157

3.112

23

24

-1

Oikos

3.136

3.094

24

25

-1

Ecography

3.066

3.132

25

23

2

Perspectives in Plant Ecology Evolution and Systematics

3.043

3.261

26

22

4

Behavioral Ecology

3.018

3.075

27

26

1

Oecologia

2.973

3.022

28

28

0

Diversity and Distributions

2.965

3.035

29

27

2

Evolutionary Ecology

2.905

2.068

30

45

-15

Biogeosciences

2.813

2.520

31

34

-3

Behavioral Ecology and Sociobiology

2.754

2.699

32

29

3

Ecosystems

2.684

2.637

33

30

3

Microbial Ecology

2.558

2.522

34

33

1

Marine Ecology-Progress Series

2.546

2.533

35

32

3

Animal Conservation

2.495

2.392

36

36

0

Ecotoxicology

2.405

1.954

37

49

-12

Aquatic Microbial Ecology

2.385

2.366

38

37

1

Agriculture Ecosystems & Environment

2.308

2.505

39

35

4

Journal of Vegetation Science

2.251

2.152

40

43

-3

Basic and Applied Ecology

2.247

2.351

41

38

3

Journal of the North American Benthological Society

2.217

2.224

42

40

2

Ecology and Society

2.215

2.049

43

46

-3

Ecological Engineering

2.175

2.330

44

39

5

Biological Invasions

2.125

2.165

45

42

3

Ecological Modelling

2.077

2.196

46

41

5

Landscape Ecology

2.061

2.624

47

31

16

Theoretical Population Biology

1.950

1.917

48

50

-2

Journal of Chemical Ecology

1.941

1.959

49

48

1

Restoration Ecology

1.928

1.993

50

47

3

Table 2: Impact Factors, Ranks, and Change in Ranks for the Top 50 Environmental Sciences Journals

Journal Title

JCR Impact Factor

Scopus Impact Factor

JCR Rank

Scopus Rank

Change in Rank

Environmental Health Perspectives

5.636

5.804

1

1

0

Global Change Biology

4.786

5.000

2

2

0

Critical Reviews in Environmental Science and Technology

4.615

4.615

3

3

0

Environmental Science & Technology

4.363

4.285

4

6

-2

Global Biogeochemical Cycles

4.335

3.947

5

8

-3

Frontiers in Ecology and the Environment

4.269

4.241

6

7

-1

Annual Review of Environment and Resources

4.036

4.357

7

5

2

Conservation Biology

3.934

3.926

8

9

-1

Global Environmental Change-Human and Policy Dimensions

3.915

4.559

9

4

5

Environmental Science and Pollution Research

3.894

3.173

10

17

-7

Ecological Applications

3.571

3.562

11

10

1

Water Research

3.427

3.223

12

15

-3

Biological Conservation

3.296

3.354

13

12

1

Applied Catalysis A-General

3.166

3.290

14

13

1

Environmental Pollution

3.135

3.231

15

14

1

Geobiology

3.114

2.864

16

21

-5

Remote Sensing of Environment

3.013

3.207

17

16

1

Journal of Environmental Science and Health Part C- Environmental Carcinogenesis & Ecotoxicology Reviews

3.000

3.429

18

11

7

Environmental Research

2.962

2.996

19

18

1

Climatic Change

2.890

2.826

20

22

-2

Environmental Chemistry

2.809

2.351

21

30

-9

Environment International

2.797

2.977

22

19

3

Chemosphere

2.739

2.820

23

23

0

International Journal of Hydrogen Energy

2.725

2.893

24

20

4

Atmospheric Environment

2.549

2.729

25

24

1

Biogeochemistry

2.534

2.592

26

26

0

Environmental Reviews

2.467

2.600

27

25

2

Ecotoxicology

2.405

1.954

28

46

-18

Environmental and Molecular Mutagenesis

2.361

2.340

29

31

-2

Aerosol Science and Technology

2.350

2.070

30

42

-12

Journal of Hazardous Materials

2.337

2.442

31

28

3

Marine Pollution Bulletin

2.334

2.368

32

29

3

Journal of Environmental Quality

2.331

2.218

33

36

-3

Environmental Toxicology and Chemistry

2.309

2.218

34

37

-3

Agriculture Ecosystems & Environment

2.308

2.505

35

27

8

Journal of Paleolimnology

2.287

2.182

36

39

-3

Science of the Total Environment

2.182

2.298

37

34

3

Ecological Engineering

2.175

2.330

38

32

6

Water Resources Research

2.154

1.914

39

48

-9

Estuaries

2.133

1.867

40

51

-11

Environmental Modelling & Software

2.099

2.293

41

35

6

Ecotoxicology and Environmental Safety

2.014

2.118

42

41

1

Journal of Industrial Ecology

1.962

2.308

43

33

10

Boreal Environment Research

1.951

2.122

44

40

4

Marine Environmental Research

1.930

1.930

45

47

-2

Journal of Aerosol Science

1.902

2.019

46

43

3

Energy Policy

1.901

2.185

47

38

9

Journal of Contaminant Hydrology

1.852

1.955

48

45

3

Journal of Environmental Monitoring

1.833

1.792

49

54

-5

Reviews of Environmental Contamination and Toxicology

1.818

1.818

50

52

-2

Conclusion

Impact factors are useful for ranking and comparing scholarly journals, and are one of the most widely used indicators of journals' significance to the scholarly community. With all the weight that is given to the impact factor, it is important that these values do not significantly vary from one database's calculation to another. This study's comparison between JCR and Scopus impact factors in the fields of ecology and environmental sciences showed no statistically significant difference, which indicates that one or the other may be used for evaluative purposes.

While most of the impact factors and journal ranks for Web of Science and Scopus were fairly similar, there were a few journals that had significantly different impact factors. In some instances, these differences could be accounted for by incomplete indexing in Scopus. Because we relied upon the number of citable articles in JCR to calculate our Scopus impact factors, our methodology may have underinflated the Scopus impact factor and rank for some titles. Our quick examination of the data in Web of Science also shows that number of cites used in JCR might be incorrect for some titles. Finally, the collection of titles indexed by the two citation databases can affect the differences in rank and impact factors seen in JCR and Scopus.

Since JCR impact factors are the standard used by most journal advertisers and university administrators, we may now be confident that these values are close enough to Scopus impact factors to justify the use of a single database's impact factor. This is valuable information because it proves that it is not necessary to calculate Scopus impact factors in addition to the JCR data. It is, however, still important to use the JCR impact factor carefully. There are many variables that can affect the impact factor of a journal that must be considered when using Garfield's ratio to evaluate the importance of a journal.

References

Bakkalbasi, N., Bauer, K., Glover, J. & Wang, L. 2006. Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomedical Digital Libraries 3(1):7.

Barschall, H.H. 1988. The cost-effectiveness of physics journals. Physics Today. 41(7):56-59.

Coleman, A. 2007. Assessing the value of a journal beyond the impact factor. Journal of the American Society for Information Science and Technology 58(8):1148-1161.

Coleman, R. 1999. Impact factors: Use and abuse in biomedical research. The Anatomical Record 257:54-57.

Garfield, W. 1972. Citation analysis as a tool in journal evaluation: Journals can be ranked by frequency and impact of citations for science policy studies. Science 178(4060):471-479.

Garfield, E. & Sher, I.H. 1963. New factors in the evaluation of scientific literature through citation indexing. American Documentation 14(3):195-201.

Hirsch, J.E. 2005. An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America. 102(46):16569-16572.

Lowry, R. 2008. VassarStats: Web Site for Statistical Computation. [Online]. Available: http://faculty.vassar.edu/lowry/VassarStats.html [July 8, 2008].

Monastersky, R. 2005. The number that's devouring science: The impact factor, once a simple way to rank scientific journals, has become an unyielding yardstick for hiring, tenure, and grants. The Chronicle of Higher Education 52(8):A12.

Opthof, T. 1997. Sense and nonsense about the impact factor. Cardiovascular Research 33:1-7.

Pislyakov, V. 2007. Comparing two "thermometers": Impact factors of 20 leading economic journals according to Journal Citation Reports and Scopus. E-LIS. [Online]. Available: http://eprints.rclis.org/archive/00011865 [July 2, 2008].

Seglen, P.O. 1997. Why the impact factor should not be used for evaluating research. BMJ 314:498-502.

Thomson Scientific. 1994. The Thomson Scientific impact factor. [Online]. Available: http://scientific.thomsonreuters.com/free/essays/journalcitationreports/impactfactor [June 27, 2008].

Werner, Y.L. 2006. The case of impact factor versus taxonomy: a proposal. Journal of Natural History 40(21-22):1285-1286.

Previous Contents Next

W3C 4.0 Checked!