Knight.indd


Analysis of Citations in  
Undergraduate Papers 1

Stacey Knight-Davis and Jan S. Sung

This paper presents the findings of a citation analysis of papers written 
by undergraduate students.The analysis included the types of materials 
cited, number of citations per paper, publication year, online availability, 
andrefereedstatusofmaterialscited.Libraryownershipofmaterialswas 
also analyzed.Number of citations in each paper increased over the first 
three papers, as did the number of refereed journals cited.There was 
also a positive correlation between the number of citations in the paper 
and the word count of the paper.

itation analysis is a useful 
method to gather information 
about the materials students 
have available when writing 

papers. Student citations offer an op-
portunity to assess library collections 
and services. A library’s collections can 
be compared to works cited to see if they 
hold the items used most o en by stu-
dents. Citations can also be analyzed to 
determine the format, date, and refereed 
status of the materials cited. This kind of 
data is useful in determining whether or 
not information literacy programs are ef-
fective in encouraging the use of scholarly 
sources. 

This project began with the desire to 
improve the collections and instruction 
services at Eastern Illinois University’s 
Booth Library by be er understanding 
which sources undergraduate students 
were citing in their papers. We dis-
covered that our campus had already 
built a depository of student papers to 
assess student writing. This collection 
included papers drawn from the entire 
undergraduate student body, with papers 

collected from the same student peri-
odically to show changes over time. This 
collection of papers is unique because it 
represents truly authentic examples of 
student writing. Students were free to 
submit any paper they considered repre-
sentative of their writing ability. As there 
were not strict criteria in place, and each 
paper was also a graded assignment, the 
papers submi ed by the students offer 
a snapshot of genuine student writing. 
This paper collection contains accurate 
representations of the papers students 
write for course credit. The study of this 
excellent collection of student work pro-
vides baseline data for future instruction 
and information literacy programs. It 
also serves to collect evidence to guide 
collection development by describing the 
types of materials students cite, their use 
of online resources, and the availability of 
materials cited in the library’s collection. 

Literature Review
Citation analysis has a long history as a 
collection development tool. Librarians 
have examined citations to determine the 

Stacey Knight-Davis and Jan S. Sung are Librarians and Assistant Professors in Booth Library at Eastern 
Illinois University; e-mail: slknight@eiu.edu and jssung@eiu.edu. 

447

mailto:jssung@eiu.edu
mailto:slknight@eiu.edu


448 College & Research Libraries September 2008

value and need for specific materials for 
decades. More recently, citation analysis 
has become a tool librarians can use to 
assess the information literacy skills of 
students. 

In 1998, Margaret J. Sylvia studied both 
graduate and undergraduate psychology 
student bibliographies to evaluate the use 
of her institution’s journal collection. Syl-
via found that 70 percent of the journals 
cited were held by the library.1 Erin T. 
Smith also performed citation analysis to 
evaluate collections, studying theses and 
dissertations. Books, periodicals, confer-
ence proceedings, theses, dissertations, 
and Web site citations were analyzed. The 
library owned 87 percent of all materials 
cited. 2 

Citation analyses performed to assess 
information literacy include the landmark 
study by Phillip M. Davis and Suzanne 
Cohen with undergraduate economics 
students.3 Davis replicated his own study 
in 2002, finding a significant increase in 
the citation of Web sites between 1996 and 
2000. He also found that library instruc-
tion alone was not effective in increasing 
the number of journal and book citations 
in student bibliographies.4 Andrew M. 
Robinson and Karen Schlegl studied bibli-
ographies from political science students. 
Their findings showed that an effective 
method to raise the number of scholarly 
sources in student bibliographies was to 
combine a library instruction session with 
an academic penalty for using nonschol-
arly sources.5 

Other studies providing data on the 
types of cited materials include Joseph 
R. Kraus’ analysis of papers presented at 
an undergraduate research symposium 
for advanced biology. Kraus found the 
majority (76.2%) of citations were to schol-
arly journals. Only 1 percent of citations 
were to Web sites.6 Charles Oppenheim 
and Richard Smith analyzed final-year 
projects from Information Science stu-
dents, finding books (40.2%) and journals 
(29.5%) to be the most heavily cited.7 Fei 
Yu, Jane Sullivan, and Leith Woodhall 
analyzed bibliographies from engineering 

students, finding that citations to Internet 
sources decrease as students progress in 
their course of study. For first-year bibli-
ographies, 67.1 percent of citations were 
to Web sites, while in bibliographies from 
fourth-year students 22.9 percent of cita-
tions were to Web sites.8 

All of the studies mentioned above 
use student work selected from a single 
discipline, course, or year of study. In 
contrast, Jake Carlson selected papers 
from classes across all disciplines that 
assigned a research paper. The research 
paper assignment required sources be-
yond class readings. Carlson found that 
the majority of sources cited (50%) were 
books. Journal articles were the next most 
commonly cited, at 19 percent, followed 
by Web sites at 16 percent. First-year 
students cited Web sites more o en than 
journals, but all other classes used journal 
articles more than Web sites.9 

Method
Student work analyzed in this study 
comes from the Electronic Writing Portfo-
lio. Each undergraduate student at East-
ern Illinois University (EIU) is required 
to periodically submit writing samples 
to build an individual portfolio. The Elec-
tronic Writing Portfolio was developed by 
the EIU Commi ee for the Assessment of 
Student Learning (CASL) to assess stu-
dent writing effectiveness. For submission 
1, students must submit a paper wri en 
in a 1000-level English course before they 
have earned 30 credit hours. Submission 
2 must come from a writing-centered or 
writing-intensive course below the 4000 
level and be submi ed when the student 
has earned between 30 and 59 credit 
hours. Submission 3 is made when the 
student has between 60 and 89 credit 
hours completed and must be from a 
3000- or 4000-level writing-centered or 
writing-intensive course. Students who 
have completed 75 hours are required to 
complete a senior seminar and submit 
work from that course to their writing 
portfolio. All submission 4 papers come 
from senior seminar. The senior seminars 



Analysis of Citations in Undergraduate Papers 449

are designed to be interdisciplinary cap-
stone courses, and students must choose 
a seminar topic outside their major area 
of study. 

When the sample used in this study 
was drawn, students could submit any 
type of work, including creative and 
technical writing, as long as work was 
submi ed from each required level. While 
students are encouraged to submit their 
best work, they are limited by the number 
and types of writing projects produced in 
each course that meets the requirements 
for each submission level. Submissions 
may only come from certain classes, so 
the documents in the portfolio become 
a snapshot of the types of assignments 
required in these classes. 

Electronic writing portfolio documents 
are stored on a server on the EIU campus. 
A program to pull random samples was 
developed locally under the guidance of 
the EIU Center for Academic Support and 
Achievement (CASA). In 2005, a sample 
of completed portfolios containing work 
submi ed between 2000 and 2005 was 
pulled to assess writing ability. Permis-
sion was granted by the CASA director for 
this sample to be used for this study also. 
The sample included 957 papers from 312 
portfolios. The average total undergradu-
ate student population from 2000 through 
2005 was 9,690 students. Not all portfolios 
contained all four submissions, as transfer 
students were generally exempted from 
the first two submission requirements. 
The year each document was submi ed 
was available in the sample data, but no 
information about the student, such as 
major or college, was provided. 

Information on sources used in student 
writing was gathered by one person. Each 
paper was available electronically as a 
separate file. Each file was opened and 
skimmed on screen to find footnotes or a 
reference list. The publication year for each 
source found was recorded. Sources were 
then coded as one of the following cat-
egories: Book, Web, Journal, Newspaper, 
Government Document, or Audiovisual. 
Sources not fi ing into these six categories 

were coded as Other. Library ownership 
of books was determined by searching 
the online catalog. Journals were checked 
against a list that included both print and 
electronic journal holdings, as well as the 
contents of aggregated databases acces-
sible through the library. Journal titles 
were further coded as Refereed or Not 
Refereed. Refereed status was determined 
using Ulrich’s Periodicals Directory. 

All identifying information on the 
student was removed from the paper by 
the CASA staff. Each paper was assigned 
an identifying number, and the year it 
was wri en was recorded. A word count 
was made for each paper using Microso  
Word. 

Data were initially recorded in a 
spreadsheet and later converted for 
analysis with SPSS (Statistical Package 
for Social Science 14.0). Data were heav-
ily treated using recoding, frequencies, 
and cross-tabulation. Statistical tests 
employed were Chi-Square, ANOVA, and 
correlation with two-tailed significance 
tests at the 0.05 level. 

Results
From the sample of 957 papers, 293 had 
no citations, footnotes, or reference list of 
any kind. Another 244 papers included 
in-text citations but no reference list or 
footnotes (table 1). We defined in-text 
citation as any bibliographic information 
inserted into the text of the document. 
This included standard in-text citations 
in APA and MLA formats, as well as par-
enthetical fragments of information. We 
encountered page numbers without a title 
or author, Web page titles, author names, 
and URLs, all of which we defined as “in-
text citations.” We defined reference list as 
any bibliographic information at the end 
of the document or as a footnote. Again, 
this category included standard citation 
formats as well as bare URLs and other 
citation fragments. 

A total of 420 papers had reference lists 
that could be analyzed. The percentage of 
papers that had no citations of any kind 
remained relatively the same over the 



450 College & Research Libraries September 2008

TABLE 1 
Number of Papers with/without Citations 

Papers Sub 1 Sub 2 Sub 3 
Papers without In-Text Citation or 
Reference List 

38 
(25%) 

64 
(33%) 

92 
(30%) 

Papers with In-Text Citations but 
without Reference List 

64 
(42%) 

54 
(28%) 

74 
(24%) 

Papers with In-Text Citation and 
Reference List 

51 
(33%) 

74 
(39%) 

140 
(46%) 

Total 153 
(100%) 

192 
(100%) 

306 
(100%) 

Chi-square=36.44, df=6, p<0.001 

Sub 4 
99 

(32%) 
52 

(17%) 
155 

(51%) 
306 

(100%) 

Total 
293 

(31%) 
244 

(25%) 
420 

(44%) 
957 

(100%) 

four submissions. However, the number 
of papers with in-text citations that lacked 
a reference list decreased steadily over 
the four submissions. For submission 1, 
42 percent lacked a reference list. Submis-
sion 2 had 28 percent with no reference 
list; and in submission 3, 24 percent had 
no reference list. For submission 4, 17 
percent of papers had no reference list. 
The differences in citing behaviors of 
students over the four submissions were 
statistically significant (Chi-square=36.44, 
df=6, p<0.001). 

Word counts (table 2) were significantly 
affected by whether students cited sources 
or not (F=65.05, df=2, p<0.001). While the 
average word count of papers without any 
form of citation was 946, average word 
count of papers with a reference list was 
1,651. The correlation coefficient between 

TABLE 2 
Word Counts and Citations 

Paper Word Counts 
N Mean SD 

Papers without In-Text Citation 
or Reference List 

288 946 831 

Papers with In-Text Citations 
but without Reference List 

243 1,350 722 

Papers with In-Text Citation 
and Reference List 

415* 1,651 833 

Total 946 1,359 858 
F=65.61, df=2, p<0.001 
*Papers without word counts, total of 5 papers, were not included. 

word counts and the number of citations 
among 415 papers with a reference list was 
r=0.554 (p<0.001). Thirty percent (r2=0.3) 
of the variation in word count can be ac-
counted for by the number of citations. 
Simply put, papers with more citations 
will typically have a higher word count. 
The word count of the reference list or 
footnotes was included in the word count 
for each paper. However, the difference in 
word count between papers with no cita-
tions of any kind and papers with a refer-
ence list is much higher than the number 
of words in the reference list. 

All of the following analyses were 
based on the 420 papers with reference 
lists. The information available in the 
papers without reference lists was too 
fragmentary and vague to be usable. 
Table 3 shows the mean number of cita-

tions, by type of source, for 
each submission number. 
The average overall cita-
tions increased over time 
from 3.57 for submission 
1 to 5.13 for submission 
3. This trend is broken 
at submission 4, where 
the average dropped to 
4.88. Differences in mean 
number of citations among 
submissions were statisti-
cally significant (F=3.22, 
df=3, p<0.02). However, at 
the individual level, only 
the difference between 

http:Chi-square=36.44


Analysis of Citations in Undergraduate Papers 451

FIGURE 1 
Number of Papers vs. Number of Citations in a Paper 

0

20

40

60

80

100

N
u
m
b
e
r
o
f
P
a
p
e
rs

2 1 1

8
3
(2
0
%
)

5
6
(1
3
%
)

5
8
(1
4
%
)

4
5
(1
1
%
)

5
3
(1
3
%
)

3
4
(8
%
)

1
9
(5
%
)

1
1
(2
%
)

1
4
(3
%
)

1
2
(3
%
)

9
(2
%
)

1
5
(4
%
)

5
(1
%
)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 16 17

Number of Citations in aPaper

3

18

1

21

submission 1 and submission 3 was sta-
tistically significant. 

Figure 1 provides more information on 
the number of citations per paper. Most 
students cited 5 or fewer sources in their 
papers. Only 37 of the 420 papers ana-
lyzed had more than 10 citations. Table 

3 breaks down the citations per paper to 
each submission. The mean number of 
citations per paper for all four submis-
sions combined was 4.42. 

Figure 2 illustrates the use of different 
source types. The books category includes 
monographs, reference books, two e-

TABLE 3 
Mean Number of Citations to Different Types of Source 

Type of 
Sources 

Submission 1 
N = 51 

Submission 2 
N = 74 

Submission 3 
N = 140 

Submission 4 
N=155 

Sum Mean SD Sum Mean SD Sum Mean SD Sum Mean SD 

Books 52 1.02 0.97 137 1.85 2.11 197 1.41 1.82 201 1.30 2.20 
Web 80 1.57 2.69 50 0.68 1.38 147 1.05 1.71 282 1.82 2.18 
Journals 36 0.71 1.68 81 1.09 2.24 278 1.99 3.59 139 0.90 2.05 
Other 11 0.22 0.50 20 0.27 0.87 36 0.26 0.80 27 0.17 0.70 
Newspapers 1 0.02 0.14 4 0.05 0.28 27 0.19 1.09 47 0.30 0.82 
Gov Doc 2 0.04 0.20 7 0.09 0.71 28 0.20 0.69 23 0.15 0.48 
Audio-Visual 0 0 0 5 0.07 0.30 5 0.04 0.22 38 0.25 0.91 
Total 
Citations 

182 3.57 3.28 304 4.11 2.96 718 5.13 3.94 757 4.88 3.49 

Differences in Number of Total Citations among Submissions: F=3.232, df=2, p<0.022 
Difference in Mean Number of Citations to Books among Submissions: F=2.11, df=3, p<0.1 (Not significant) 
Difference in Mean Number of Citations to Web among Submissions: F=7.0, df=3, p<0.001 
Difference in Mean Number of Citations to Journals among Submissions: F=5.28, df=3, p<0.001 



452 College & Research Libraries September 2008

FIGURE 2 
Percentage of Papers with Certain Type of Sources by Submission Number 

Actual number of papers citing certain types of sources was denoted in parentheses. For example, 
65% or 33 papers submitted during submission 1 cited book(s). 

N=51 N=74 N=140 N=155 

books, and an electronic encyclopedia. 
Web is defined as electronic documents 
that are not electronic books, journals, 
newspapers, or government documents. 
Journals include both print and elec-
tronic serial publications that are not 
newspapers or government documents. 
Magazines are included in the journals 
category. The other category includes 
annual reports, personal interviews, 
brochures, instruments, and works for 

which a source type could not be deter-
mined due to lack of information in the 
citation. Newspaper includes both print 
and electronic newspapers. The Gov Docs 
category also includes both print and elec-
tronic documents. Audiovisual includes 
films, television shows, recordings of 
songs, and two-dimensional artwork. 

Books were the preferred source for 
submissions 1, 2, and 3. The percentage of 
papers with book sources increased from 

TABLE 4 
Number of Papers with Citations to a Single Source and Papers with 

Different Types of Sources in the Same Paper 
Number of 
Source Types 

Submission 
1 

Submission 
2 

Submission 
3 

Submission 
4 

Total 

1 29 (57%) 39 (53%) 72 (51%) 67 (43%) 207 (49%) 
2 17 (33%) 26 (35%) 35 (25%) 49 (32%) 127 (30%) 
3 5 (10%) 7 (9%) 24 (17%) 29 (19%) 65 (15%) 
4 0 2 (3%) 9 (7%) 8 (5%) 19 (5%) 
5 0 0 0 2 (1%) 2 (1%) 
Total 51 (100%) 74 (100%) 140 (100%) 155 (100%) 420 (100%) 
Chi-Square=15.63, df=12, p<.21 (Not significant) 



Analysis of Citations in Undergraduate Papers 453

TABLE 5 
Articles Available Online by Submission Number 

Online 
Availability 

Submission 
1 

Submission 
2 

Submission 
3 

Submission 
4 

Not 
Available 
Online 

18 (50%) 33 (41)% 63 (23%) 24 (17%) 

Available 
Online 

18 (50%) 48 (59%) 215 (77%) 115 (83%) 

Total 36 81 278 139 
Chi-Square=27.15, df=3, p<0.001 

Total 

138 (26%) 

396 (74%) 

534 

submission 1 to submission 2 and decreased 
for later submissions. Students were more 
likely to cite Web sources than books for 
submission 4 from the senior seminar. Cita-
tions to Web sources were more common 
than journals in first-submission papers, but 
the trend reversed for second- and third-
submission papers. Citations to journals 
steadily increased over time except for 
submission 4. There was no statistically 
significant difference in the average number 
of citations to books over time (F=2.11, df=3, 
p<0.1). In contrast, the average number of 
citations over time to Web or to journals was 
statistically significantly different (F=7.0, 
df=3, p<0.001 for Web, and f=5.28, df=3, 
p<0.001 for journals). 

Table 4 shows the number of papers 
with citations to a single source type 
and the number of papers with multiple 

source types in the same paper. Of the 
420 papers with a reference list, nearly 50 
percent of the papers cited only a single 
source type. Among 207 papers with a 
single source type, almost half of them, or 
99 papers, cited only books. Web sources 
were the only source for 46 papers, and 
38 papers cited only journal sources. The 
number of papers with a single source 
type decreased over time from 57 percent 
for submission 1 to 43 percent for submis-
sion 4. The use of multiple source types 
increased over time, but the difference 
did not reach statistical significance (Chi-
Square=15.63, df=12, p<0.21). 

For all source types combined, 57 per-
cent of citations were to materials avail-
able online. These materials included Web 
sites as well as newspapers (60 online/79 
total), books (5 online/587 total), govern-

TABLE 6 
Publication Year by Source Type 

Source 
Type 

N/A -1950 1951-
1960 

1961-
1970 

1971-
1980 

1981-
1990 

1991-
2000 

2001-
2005 

Total 

Books 15 18 13 33 57 71 273 107 587 
Journals 10 0 3 16 17 32 245 211 534 
Web 296 0 0 3 2 3 88 167 559 
Newspaper 5 0 0 0 0 4 11 59 79 
Gov Doc 15 0 0 0 2 5 19 19 60 
Audio-
visual 

4 0 3 0 1 6 21 13 48 

other 66 0 0 0 0 0 14 14 94 
Total 411 18 19 52 79 121 671 590 1961 

http:Square=15.63


454 College & Research Libraries September 2008

FIGURE 3 
Percentage of Papers with Certain Type of Sources by Submission Number 

30 (27%)25 (69%)

64 (23%)

89 (64%)

50 (36%)

214 (77%)

51 (63%)

11 (31%)

0

50

100

150

200

250

300

350

Submission 1 Submission 2 Submission 3 Submission  4

Not Refereed Refereed

N
u
m
b
e
r
o
f
P
a
p
e
rs
w
it
h
C
it
a
ti
o
n
s
to
A
rt
ic
le
s

208 (39%)

326 (61%)

Total

SubmissionNumber

ment documents (40 online/60 total), and 
journals available on the Internet (396 
online/534 total). For journals, 396 articles, 
or 74 percent out of 534 articles cited, were 
available online. Online availability of 
articles increased steadily, from 50 percent 
for submission 1 to 83 percent for the last 
submission from the senior seminar (table 
5). It is not known if students used the on-
line version of these materials or the print 
version. It is only known that the online 
version was available to them. 

Table 6 shows the pa ern of publica-
tion years and source types. For all source 
types, students tended to use more cur-
rent materials. More than 65 percent of 
books and 85 percent of articles cited were 
published a er 1991. 

Overall, students cited refereed jour-
nals more o en than nonrefereed, with 
61 percent of journal citations to refereed 
titles (figure 3). The use of refereed articles 
doubled from 31 percent for submission 
1 to 63 percent for submission 2. The per-
centage increased again to 77 percent for 
submission 3 but decreased to 36 percent 
for submission 4. This difference was 
statistically significant (Chi-Square=80.63, 
df=3, p<0.001). 

Figure 4 and figure 5 show whether 
the library held the books and articles 

cited by the students. Overall, 55 percent 
of cited books were held by the library. 
Many of the cited books not owned by the 
library were course textbooks, which the 
library does not generally include in its 
collection because all students are issued 
textbooks from the university textbook 
rental service. Journals, which are typi-
cally available only through a library, are 
a be er indicator of the match between 
library holdings and student use. The 
library offered access to 80 percent of the 
articles students cited. 

Discussion
It is clear that the submission number 
of the paper, which is a reflection of the 
course taken and the amount of time the 
student has been in college, is related 
to the type and amount of material that 
students cite. In general, we had expected 
that the number of reference list citations 
per paper would increase from submis-
sion 1 to submission 4, but this was not 
the case. While it is difficult to completely 
establish which factors most heavily in-
fluence student citation behavior, some 
trends are clear. 

The number of papers that included a 
reference list did increase over the four 
submissions, as expected. For submission 

http:Chi-Square=80.63


Analysis of Citations in Undergraduate Papers 455

FIGURE 4 
Ownership of Books Cited 

0

20

40

60

80

100

120

N
u
m
b
e
r
o
f
B
o
o
ks
C
ite
d

3 (2%)
6 (4%)

3 (6%)

84 (42%)

46 (34%)

25 (48%)

110 (56%)

85 (62%)

24 (46%)

Submission 1 Submission 2 Submission 3

Not known Not owned Owned

9 (4%)

89 (44%)

103 (51%)

Submission 4

1, only 33 percent of papers had a refer- The large number of papers without 
ence list, but submission 4 had a reference citations was disappointing when first 
list in more than 51 percent of the papers. discovered. This result was partially ex-
This aspect of citation behavior has a plained by the CASA staff. Students are 
simple relationship to the amount of time encouraged to submit their best writing, 
the student has spent in school. The more not their best research. Many students 
hours a student has completed, the more choose to submit essays, creative writing, 
likely he or she is to cite sources. or reflective pieces. CASA staff also ex-

FIGURE 5  
Ownership of Journals Cited  

1
7
(1
2
%
)

2
5
(9
%
)

1
3
(1
6
%
)

2
(6
%
)

1
2
(9
%
)

1
9
(7
%
)

1
0
(1
2
%
)

9
(2
5
%
)

1
1
0
(7
9
%
)

2
3
4
(8
4
%
)

5
8
(7
2
%
)

2
5
(6
9
%
)

0

50

100

150

200

250

Submission 1 Submission 2 Submission 3 Submission 4

Notknown Notowned Owned

N
u
m
b
e
r
o
f
A
rt
ic
le
s
C
it
e
d



456 College & Research Libraries September 2008

plained that some students choose to keep 
their works-cited list as a separate file. 
Students are not required to combine the 
works-cited file with the file containing 
the body of the paper before submi ing, 
which resulted in a group of papers with 
in-text citations but no reference list. 

In addition to the positive relationship 
between including a reference list and 
number of class hours completed, the 
analyses performed here also show a posi-
tive relationship between paper length 
and word count. While the idea that 
longer papers would cite more sources 
is intuitive, it is reassuring to see that 
this idea holds up with a large sample of 
papers from many disciplines. Averaged 
across all submissions, students who cited 
sources and had a reference list wrote 705 
more words than those who did not cite 
sources and did not include a reference 
list. The reference lists were included in 
the word count. It is estimated that the 
typical citation in a reference list for pa-
pers in this study was 19 words. 

As noted in other studies, books were 
the preferred source for many students. 
For all submissions combined, books ac-
counted for 59 percent of citations. This 
is comparable to Carlson’s results10 of 
50 percent of citations to book sources. 
However, the mean number of citations 
per paper found by Carlson was much 
higher. In this study, mean number of cita-
tions for submission 1 was 3.57. The mean 
for submission 4 was 4.88. In comparison, 
Carlson’s first-year student papers had 
a mean of 7.78 citations per paper, and 
senior papers had a mean of 10.11. This 
difference is most likely due to Carlson’s 
sampling method, which selected papers 
only from classes assigning a research 
paper. 

The number of citations to journals was 
highest for submission 3, with a mean of 
1.99. This is lower than Carlson’s finding11 
of a mean of 2.57 citations to journals in 
papers wri en for 3000- and 4000-level 
courses. However, Carlson found that the 
mean number of citations to journals in 
papers by juniors in the humanities was 

only 0.43. As the exact proportions for the 
number of students in each discipline was 
not known in this study, it is not possible 
to determine whether the relatively low 
number of journal citations in 3000- and 
4000-level courses is due to discipline-
specific citation practices or some other 
factor. 

Use of refereed journals increased 
sharply for submission 3, with 77 percent 
of article citations to refereed journals. 
Since the courses from which students 
can submit submission 3 typically require 
a research paper, the sharp increase in 
use of refereed articles is likely due to re-
quirements in the paper assignment. This 
assumption is borne out by the sharp de-
crease in journal usage for submission 4. 

The citation pa erns seen in submis-
sion 4 papers do not follow the trends 
established in the first three submissions 
(graph 2). Journal citations drop off (28% 
for submission 4, 42% for submission 3), 
and citations to Web sites rise (61% for 
submission 4, 38% for submission 3). Ci-
tations to audiovisual materials also rise 
(13% for submission 4, 3% for submission 
3). The number of citations to nonrefereed 
journals increased as well (64% for sub-
mission 4, 23% for submission 3). 

All papers for submission 4 come from 
senior seminar classes. Two circumstantial 
factors were identified that may contribute 
to the citation pa erns seen in papers from 
these classes. First, a study of writing ef-
fectiveness conducted by CASA on this 
same sample found that senior papers 
were of lesser quality than papers from 
other classes.12 The second factor is the 
lack of a coordinated library instruction 
program for senior seminar students. Rob-
inson and Schlegl found that instruction 
together with enforced instructor guide-
lines are an effective way of generating 
quality student bibliographies.13 

Considering these two factors, we 
suspect that the lack of standardization 
in writing requirements for the senior 
seminar classes is responsible for the cita-
tion pa ern seen in submission 4 papers. 
The goal of all senior seminar courses is 

http:bibliographies.13
http:classes.12


Analysis of Citations in Undergraduate Papers 457

to provide a “cross-disciplinary culminat-
ing experience that will provide students 
with an opportunity to apply concepts 
and use skills developed in both their 
general education and major courses,”14 
and all courses must include a writing 
component. There are requirements on 
the percentage of the grade that must be 
based on writing and also a requirement 
that at least one paper must be revised, 
but there is no requirement on the length 
of those papers. 

The wide range of topics and teach-
ing styles used in these courses leads 
to a huge variability in the assignments 
given, and the lack of standards for the 
length of writing assignments allows for 
a large variation in the length of papers 
submi ed. While all courses must address 
literacy, critical thinking, and citizen-
ship, library research is not specifically 
required. Assessment of these courses is 
carried out by the instructor, based on his 
or her set learning objectives. 

It is possible to argue that students 
simply revert to more basic Web search-
ing and more basic magazine articles 
when faced with material from outside 
their major in senior seminar. However, 
the poor writing quality found in these 
courses suggests a problem with the as-
signment or poor revision. Since only one 
assignment must be revised, it is possible 
that students submit assignments that did 
not require revision. 

Conclusion
This analysis of student citation behavior 
reaffirmed some of the assumptions we 
had made about our collections. It also 
gave us further insight into the types of 
materials students cite and how many 
citations their papers typically include. 
Having this study completed allows 
Booth Library to make decisions based 
on the evidence of past student behavior 
instead of anecdotal observances and 
assumptions. 

One of the assumptions confirmed by 
this study is the heavy use of materials 
that are available on the Internet. Over 

all four submissions, 74 percent of the 
articles cited were available online. For 
submission 4, 83 percent of articles cited 
were available online. We are pleased to 
note that eleven journals cited by students 
in print were later licensed in electronic 
form. Having evidence of undergraduate 
students’ strong preference for electronic 
content is helpful when discussing elec-
tronic-only journal subscriptions with the 
teaching faculty. The large percentage of 
online material cited in student papers 
also provides support for licensing elec-
tronic content. 

In addition to providing baseline data 
on citation behavior, this project also 
provided important insight into current 
library instruction programs at our uni-
versity. At this time, library instruction 
is offered to any instructor requesting 
it. Each instructor determines type and 
amount of library instruction students 
receive. A dialogue has begun between 
the library and academic departments to 
determine if the current system is meet-
ing the needs of students and instructors. 
Should changes be made in curriculum 
or in the way library instruction is of-
fered, this study will provide valuable 
baseline data to assess the effectiveness 
of the change. 

An important point to be taken from 
this study is the importance of the re-
quirements of the paper assignment on 
citation behavior. The sources a student 
will choose to cite are o en heavily in-
fluenced by the requirements regarding 
sources, or lack thereof, in the paper as-
signment. Librarians typically see only 
students who are actively seeking sources 
and may overlook students who have not 
had classes that required them to conduct 
research or cite sources. It is important 
to remember this diversity of student ex-
perience when conducting bibliographic 
instruction sessions and helping students 
in the reference room. 

Another benefit from this project was 
that it allowed Booth Library to develop a 
good working relationship with the people 
and commi ees responsible for student 



458 College & Research Libraries September 2008

assessment on our campus. The library 
has also become more involved in campus 
assessment activities. A library faculty 
member now belongs to the Commi ee 
for the Assessment of Student Learning 
(CASL). We hope that this involvement 
will ensure that the library’s assessment 
needs are considered as changes to the 
Electronic Writing Portfolio procedures 
are discussed. A major limitation of this 
study was that no information on academ-
ic discipline or college was available. We 
hope that, by having a library representa-
tive at CASL, we can encourage changes 
that permit this data to be collected. 

While this study describes the citation 
behavior of students at one university, all 
libraries should take an interest in what 
students at their institutions are writing 
and how well the library is supporting 
student needs. While citation analysis 
can show us what students choose to 
cite, additional research is needed to 
determine why students choose these 
sources. It is imperative that all librar-
ians and information professionals 
continually examine the behavior of 
their users and clients to ensure that 
the best possible services and resources 
are provided. 

Notes

1. Margaret J. Sylvia, “Citation Analysis as an Unobtrusive Method for Journal Collection 
Evaluation Using Psychology Student Research Bibliographies,” Collection Building 17, no. 1 (1998): 
20–28. 

2. Erin T. Smith, “Assessing Collection Usefulness: An Investigation of Library Ownership 
of the Resources Graduate Students Use,” College & Research Libraries 64 (Sept. 2003): 344–55. 

3. Phillip M. Davis and Suzanne A. Cohen, “The Effect of the Web on Undergraduate Citation 
Behavior 1996–1999,” Journal of the American Society for Information Science and Technology 52 (Feb. 
15, 2001): 309–14. 

4. Phillip M. Davis, “The Effect of the Web on Undergraduate Citation Behavior: A 2000 
Update,” College & Research Libraries 63 (Jan. 2002): 53–60. 

5. Andrew M. Robinson and Karen Schlegl, “Student Bibliographies Improve When Profes-
sors Provide Enforceable Guidelines for Citations,” portal: Libraries and the Academy 4 (Apr. 2004): 
275–90. 

6. Joseph R. Kraus, “Citation Pa erns of Advanced Undergraduate Students in Biology, 
2000–2002,” Science & Technology Libraries 22, no.3/4 (2002): 161–79. 

7. Charles Oppenheim and Richard Smith, “Student Citation Practices in an Information 
Science Department,” Education for Information 19 (Dec. 2001): 299–323. 

8. Fei Yu, Jane Sullivan, and Leith Woodhalll, “What Can Students’ Bibliographies Tell Us? 
Evidence Based Information Skills Teaching for Engineering Students,” Evidence Based Library 
and Information Practice 1, no. 2 (2006): 12–22. 

9. Jake Carlson, “An Examination of Undergraduate Student Citation Behavior,” The Journal 
of Academic Librarianship 32 (Jan. 2006): 14–22. 

10. Ibid. 
11. Ibid. 
12. Eastern Illinois University, Electronic Writing Portfolio Readings Report Fall 2005 (Charleston, 

Ill.: The University, 2005). 
13. Robinson and Schlegl, portal: Libraries and the Academy (Apr. 2004): 275–90. 
14. Eastern Illinois University, SENIOR SEMINAR MISSION (Charleston, Ill.: The University, 

2001). Available online from www.eiu.edu/~eiucaa/SenSemMission.pdf. [Accessed 13 November 
2007]. 

www.eiu.edu/~eiucaa/SenSemMission.pdf