Materials Used by Master's Students in Engineering and Implications for Collection Development: A Citation Analysis

Virginia Kay Williams
Assistant Professor/Assistant Collection Development Officer
gwilliams@library.msstate.edu

Christine Lea Fletcher
Assistant Professor/Government Documents and Microforms Reference Librarian
cfletcher@library.msstate.edu

Mississippi State University
Mississippi State, Mississippi

Abstract

The purpose of this study is to determine the materials used by graduate students in engineering and to guide library collection development decisions. A literature review shows citation studies are useful in developing core journal lists and determining appropriate allocations between journals and monographs. This case study found that journals (38%), conference papers (19%), and books (18%) are the most heavily used formats, with books aging more slowly than other formats. Core journal lists are developed by total citations and by number of citing authors. Variations among engineering disciplines were identified, including variations in format, age, and subject classification of journals. Areas for further research are noted.

Introduction

Effective collection development requires an understanding of the research needs of patrons. Ideally, a librarian serving as a subject bibliographer or collection development liaison would have an academic background or professional expertise in the field in order to have an in-depth understanding of faculty and student research needs. Unfortunately, matching the librarian's background to the department is not always possible. Librarians who have been assigned responsibilities in unfamiliar disciplines may peruse course catalogs, department web pages, and introductory texts to gain background. Faculty research interests may be identified from lists of faculty publications, faculty web pages, and faculty requests for material purchases. Research needs of graduate students can be more difficult to discern, but citation analysis of master's theses can provide insight into the research needs of graduate students.

At Mississippi State University (MSU), a land grant institution with a strong engineering program, library faculty serve as liaisons to each discipline. None of the library faculty has a background in engineering. The teaching faculty are heavily involved in collection development and their selections generally reflect the research interests of the faculty. In 2004, the MSU Libraries participated in the Association of Research Libraries' LibQual+^TM survey to assess and improve user services. Noting that graduate students expressed higher expectations for collections than did faculty on the survey, the authors decided that more attention should be paid to graduate students' research needs.

This citation study of masters' theses identifies the research materials used by graduate engineering students in the disciplines of aerospace, agricultural and biological, chemical, civil, computational, electrical and computer, industrial, and mechanical engineering. The authors analyzed the citations to answer five research questions:

1. Which material formats are used in engineering? Material allocation formulas often include a factor for the relative importance of journals versus monographs, so awareness of graduate citation patterns may assist in developing formulas.
2. What is the age of the cited items? The age of materials is useful in developing guidelines for retrospective collection development and storage.
3. Which are the most frequently cited journals in engineering? Citation data may be used in developing core title lists of journals to maintain when flat or shrinking budget allocations require journal cancellations.
4. To what extent do masters' students in engineering use journals from non-engineering disciplines? Subscriptions to heavily cited non-engineering titles may need special consideration during journal cancellation projects.
5. How do citation patterns vary among engineering disciplines? Understanding variations can assist librarians in determining which resources might interest each department and aid in making assignments to librarians who must serve as liaison for several departments.

The purpose of this user citation study is to enhance liaisons' abilities to select materials and evaluate journals by providing data on formats and subjects used in graduate research and by identifying citation pattern variations that exist among engineering disciplines.

Literature Review

Citation analysis is a non-intrusive method of finding patterns in a specific population's use of research materials. The underlying assumption is that citations reflect the library needs of the citing authors. Several overviews of citation analysis have discussed both its uses and limitations. Broadus (1977) notes that while collection development ideally should be performed by a librarian who is also an expert in the subject field, citation analysis can provide useful guidance in the absence of such experts. Linda C. Smith (1981) suggests that citation analysis of theses and dissertations can serve as a user study with implications for both collection development and user service design. She cautions librarians to remember that citation does not imply the quality or importance of the material cited. Devin and Kellogg (1990) discuss the use of citation analysis to determine allocation of monograph and serials funds under the assumption that the proportion of funds allocated to each format should equal the proportions of citation by researchers. Garfield (1977b) states that citation frequency can determine journal selections for core collections, but is insufficient for determining all selections. Line (1978) argues that ranked journal lists from citation studies should not be the basis of purchasing decisions but may be useful for identifying journals worthy of closer examination by librarians familiar with local needs.

Citation analysis studies compiled from the professional engineering journal literature date back to 1939. The first, Patterson's (1945) analysis of cited references in the journal Industrial and Engineering Chemistry, found that even though engineering researchers cited books and older works more than did researchers in pure sciences, they still relied heavily on periodical literature and current materials. Coile's (1952) study of citations in electrical engineering journals was the first to rank engineering journals based on citations. Both Coile's 1952 study of key journals in electrical engineering and his later 1969 study of IEEE periodicals provided evidence supporting Bradford's Law of Scatter which states that, for any given subject, a small number of journals will produce a large percentage of total citations. In his studies, Coile found that a core list of 18 journals provided half of the citations in electrical engineering. Eugene Garfield (1977c; 1977a) published two studies of engineering journal citations from the Institute for Scientific Information (ISI) database that supported Bradford's Law. His study also found that engineering researchers cited books more heavily than journals and that books cited tend to be older than journals cited. Two studies of Korean researchers, one in electrical engineering and the other in mechanical engineering, found that engineering researchers cited journals most, followed by books, and then conference proceedings (Rieb 1993; Kim 2002). In their study of citations from 17 professional association journals, Musser and Conkling (1996) compared citations by researchers in specific engineering disciplines and discussed the implications of formats cited and material age on collection development, library services, and off-site storage decisions.

Few citation analysis studies exist for engineering theses and dissertations. In Erin T. Smith's (2003) comparison of citations in theses and dissertations from 1991 and 2000, science and engineering are compared to other broad subject areas. Kushkowski et al. (2003) sampled citations in theses and dissertations from 1973 to 1992 to compare the format and age of materials used in various disciplines, including engineering. Kriz (1977; 1978) analyzed citations in masters' theses to determine the extent of journal usage by different engineering disciplines as well as to determine allocations to monograph and serials budgets. He concluded that journal subscriptions should be cancelled to maintain an adequate monograph allocation. In a follow-up study, Kriz (1984) examined journal usage in dissertations by engineering discipline to identify core journals and again concluded that protecting journal subscriptions at the expense of monograph allocations is inappropriate in engineering.

Methods

The authors identified engineering master's theses accepted at MSU from 2000-2004 for inclusion in this study. Two hundred fifty theses, with a total of 9,340 citations, represented eight engineering disciplines: aerospace, agricultural and biological, civil, chemical, computational, electrical and computer, industrial, and mechanical. Table 1 shows the number of theses and citations from each discipline. The authors used Microsoft Access to create a relational database with three tables: source documents, citations, and journals. The source documents table included the source number assigned to each thesis, author, title, discipline, and year of acceptance for each thesis. The citations table included source number, citation number, material format, publication year, and journal title (if applicable) for each citation. The journals table included title, ISSN, LC classification, descriptive titles of the LC classes and subclasses, and library ownership.

Each citation's material format was recorded as book, conference paper, dissertation, federal government document, state government document, federal government technical report, journal, patent, personal communication, product literature, software, non-government technical report, technical standard, thesis, university extension publication, web site, other, and undetermined. When the material format was not clear from the citation, the authors searched OCLC WorldCat, IEEE Xplore, EI Compendex and Google for further information. Journal titles, ISSNs, and LC classification as determined from WorldCat records were entered in the journal table for each journal. When the WorldCat record did not include an LC classification, the authors assigned the LC classification appropriate to the first subject entry in the bibliographic record. For each journal, the descriptive titles of the LC class and the first subclass were added from LC schedules.

Formats Used

Materials were grouped into eight format categories for analysis: journals (serials other than monographic series and conference proceedings), books, conference papers, web resources, technical reports and standards (including government technical reports), government documents (state, federal and foreign), theses and dissertations, and miscellaneous (patents, personal communications, product literature, software and software manuals, university extension documents, unpublished materials, and other). Thirty-eight percent of the total citations were to journals, 19% percent to conference papers, 18% to books, 9% to web resources, 6% to technical reports and standards, 3% to government documents, 3% to theses and dissertations, and 6% to miscellaneous formats. Previous studies which have ranked material formats also found that journals were the most commonly cited format, while rankings of other formats varied widely (Patterson 1945; Coile 1969; Rieb 1993; Kim 2002; Kushkowski et al. 2003; Musser and Conkling 1996).

In this study, rankings of formats varied by discipline. Journals were the most cited format overall, but the percentage ranged from a low of 26% in civil engineering to a high of 58% in agricultural and biological engineering. Electrical and computer engineering was the only discipline in which journals were not the most heavily cited format; students cited a higher percentage of conference papers. As Table 2 shows, the percentage of formats used in each discipline varies greatly. Two earlier studies also found that the percentage of formats used varies by discipline. In his study of master's theses, Kriz (1978) found journal use ranged from 11% in aerospace engineering to 49% in chemical engineering. In their study of association journals, Musser and Conkling (1996) found that journal use varied from 23% in mining to 75% in bioengineering, monographs from 7% in electrical to 27% in mining engineering, and conference papers from 4% in mining to 36% in electrical engineering.

Material Age

Understanding the extent to which library users rely on older materials can be useful in determining which materials can be moved to remote storage (Ackerson 2001). Musser and Conkling (1996) and Kushkowski et al. (2003) found that the majority of materials cited were less than eight years old. In the current study, 50% of all materials cited were published within the last eight years and 80% within the last 15 years. Although the oldest item cited was a conference paper, conference papers tended to be more current than books or journals with 76% published within 10 years. Half of conference papers were five years old or less, while only 35% of books and journals fit that category. Although engineering relies heavily on current materials, older materials remain valuable; 16% of books were more than 20 years old. This supports Musser and Conkling's (1996) recommendation that conference papers could be stored or weeded after fifteen years, but books should not be placed in storage as quickly as other formats.

The age of materials cited also varied by engineering discipline. Older materials were cited in mechanical, aerospace, and chemical engineering, while more current materials were cited in industrial engineering. Table 3 shows the age of materials for all formats, books, conference papers, journals, and for each discipline.

Journals

Garfield (1977b) suggested that rankings developed from total citation counts could be a variable in developing core journal lists. However, our data indicates that when using master's theses for local citation studies, librarians should consider the number of authors citing a journal as well as the total citation count. Since multiple citations of a title by a single thesis author can affect the ranking of journals significantly, the MSU study ranked journals by the number of citing authors, as well as the total number of citations. For example, Environmental Science and Technology was the journal cited most often by total citation count, but when journals were ranked by number of citing authors AIAA Journal ranked number one. In addition, two of the top 40 journals ranked by total citation count were each cited by only one author; Carbon was cited 25 times and PCI Journal 22 times. Journals cited by seven or more authors are ranked by number of citing authors in Table 4, while journals cited at least 16 times are ranked by total citations in Table 5. Approximately one-third of the titles in each table do not appear in the other ranking. This data supports both Line's (1978) and Smith's (1981) cautions against using ranked lists from citation studies as the sole criteria for making collection decisions.

As previously discussed, journals were the format most commonly cited. Cited articles came from a total of 993 journal titles. Kriz (1977) found that Bradford's Law of Scatter applied in his study; half of all journal citations came from only 33 journals (11% percent of the total journals cited), and 70% of the journals were cited by only one author. Our study also confirms Bradford's prediction; half of all journal citations came from 95 titles (9.6% of the total journals cited), and 64% of journals were cited by only one author. Such user data should be considered in evaluating journals.

The authors determined local holdings and access by searching each title in the online catalog. The MSU Libraries have either print or electronic holdings for 82% of the journals cited; only 14% of citations were from journals that the Libraries do not hold. The authors compared holdings to the top-ranked journals shown in Tables 4 and 5. The MSU Libraries hold all titles in Table 4 and all but three titles listed in Table 5: Oil and Gas Journal, Biomaterials, and Journal for Materials Processing Technology. These three journals should be considered as possible additions during any upcoming journal evaluation project at MSU.

Both Musser and Conkling (1996) and Kriz (1978) studied formats used across the individual disciplines within engineering, but neither discussed journal rankings by discipline. In the current study, journals were ranked for each discipline by the number of citing authors and total number of citations. Tables 6 and 7 show the top-ranked journals for each discipline by citing authors and total citations respectively. In only three of the eight disciplines was a journal ranked highest by both methods: aerospace, computational, and mechanical engineering. With the exception of industrial engineering, the top journals for each discipline also appeared in the lists of highly ranked journals overall (Tables 4 and 5). Manufacturing Systems, the journal with the most citations in industrial engineering theses, was not among the forty most cited journals overall. Likewise PM Net Work, the journal cited by the most industrial engineering thesis authors, was not among the top journals cited by authors overall. Librarians need to use ranked lists cautiously, considering variations in ranking by total number of citations versus by citing authors, by broad field, and by individual disciplines within the field.

Librarians also need to consider the importance of non-engineering journals to engineering students when journal evaluation projects are planned. Patterson (1945) found that one-third of journals cited in Industrial Engineering and Chemistry were from the pure sciences. Kriz (1984) noted without providing statistics that many of the journals cited in engineering dissertations were from agriculture, medicine, mathematics, and physics. In the MSU study, 44% of titles were classed in technology, 32% in science, 10% in medicine, 7% in social sciences, 3% in agriculture, and 4% in other classes. This supports both Patterson's (1945) and Kriz's (1984) findings that journals from other subject areas are important to engineering authors. Kriz (1984) indicated that mathematics and physics were heavily cited in doctoral dissertations, but the three most heavily used subclasses of science journals in the MSU study were chemistry, mathematics, and physics. Table 8 shows the percentage of journals and of citations by LC classification.

The LC classification of journals cited varies by engineering discipline. Although technology journals were used by authors from all departments, the prevalence of use differs greatly; 77% of aerospace engineering journal citations were to technology journals while only 16% of agricultural and biological engineering citations were to technology journals. Two departments used more journals classed in pure science than in technology; 59% of journals cited by computational engineering authors and 36% cited by agricultural and biological engineering authors were classed in science. Thirty-four percent of the titles used by agricultural and biological engineering authors were classed in medicine, including the top three journals for that discipline, while 10% were classed in agriculture. Although only 7% of the journals cited overall were classed in social sciences, 28% of those used by industrial engineering authors were in that class; 42% of the journals in the subclass industries/land use/labor and 31% in commerce. The interdisciplinary nature of engineering needs to be considered when journals are evaluated.

After finding that more than half the journals cited in engineering doctoral dissertations were not coded to engineering departments, Kriz (1984) concluded that it is more important to maintain core collections in other disciplines than to expand the engineering journal collection. At MSU, journals are coded to specific departments for collection management purposes. Approximately half of the top-ranked journals (shown in Tables 4 and 5) were coded to non-engineering departments, further indicating that engineering master's students at MSU rely heavily on journals coded to departments outside the field of engineering. The data on subject classification and departmental coding indicate the need for engineering liaisons to become familiar with heavily cited non-engineering journals and to advocate for retaining important titles coded to other departments during journal evaluation projects.

Conclusions and Directions for Future Research

This study updates and expands on previous citation analysis studies that focused on engineering, especially those completed by Kriz (1977, 1978, 1984), by analyzing all citations from 250 master's theses published at a land grant university between 2000 and 2004. The results support previous findings that journals are the most commonly cited format. As in Kriz's 1978 study, this study found that a lower percentage of the citations in master's theses are to journal articles than has been found in studies of professional engineering literature. This demonstrates that collection needs of master's students can differ markedly from those of professionals and faculty. Although journals were the most commonly cited format overall, they comprised more than forty percent of citations in only two of the eight disciplines studied. The data also shows that books retain their value for research longer than journals and conference proceedings. As rising serials costs squeeze budgets, libraries will need to monitor and adjust their serials/monographs budget ratios to ensure that the library builds collections of conference papers and books which support the research needs of graduate students. Librarians who work with engineering disciplines should also ensure that graduate students are aware of and have access to other useful materials, such as government documents and technical reports.

The results of this study suggest three areas in need of future research. First, while journal rankings are usually based on total number of citations, ranking by number of citing authors may be more useful for studies of master's theses and doctoral dissertations. Second, the LC classification of journals cited indicates significant variations in subject classes cited among engineering disciplines. Further research would be useful to determine if variations in subject resources among disciplines can be generalized beyond Mississippi State University and with formats other than journals. Third, this study considers only collection development implications, but further study might suggest implications for reference, instruction, and outreach.

Acknowledgement

The authors would like to thank Deborah Lee, June Schmidt, and Zoltan Takacs for reviewing the draft of this article.

References

Ackerson, Linda G. 2001. Is Age an Appropriate Criterion for Moving Journals to Storage. Collection Management 26 (3):63-76.

Broadus, Robert N. 1977. The Applications of Citation Analyses to Library Collection Building. In Advances in Librarianship, edited by M. J. Voigt and M. H. Harris. New York: Academic Press.

Coile, R. C. 1952. Periodical Literature for Electrical Engineers. Journal of Documentation 8 (4):209-226.

________. 1969. Information Sources for Electrical and Electronics Engineers. IEEE Transactions on Engineering Writing and Speech EWS-12 (3):71-78.

Devin, Robin B., and Martha Kellogg. 1990. The Serial/Monograph Ratio in Research Libraries: Budgeting in Light of Citation Studies. College and Research Libraries: 46-54.

Garfield, Eugene. 1977a. Characteristics of Highly Cited Publications in the Engineering Sciences. In Essays of an Information Scientist. Philadelphia: ISI Press. Originally published in Current Comments, No. 12, March 22, 1976.

________. 1977b. Is Citation Frequency a Valid Criterion for Selecting Journals? In Essays of an Information Scientist. Philadelphia: ISI Press. Originally published in Current Comments, April 5, 1972.

________. 1977c. Journal Citation Studies. 21. Engineering Journals. In Essays of an Information Scientist. Philadelphia: ISI Press. Originally published in Current Comments, No. 27, July 7, 1975.

Kim, Mee-Jean. 2002. Citation Patterns of Korean Physicists and Engineers: Differences by Type of Publication Source and Type of Authorship. Scientometrics 55 (3):421-436.

Kriz, Harry M. 1977. Citation Counting and the Future of Engineering Libraries. Engineering Education 67 (7):707-710.

________. 1978. Subscriptions vs. Books in a Constant Dollar Budget. College and Research Libraries 39 (2):105-109.

________. 1984. Library Management Implications of Journal Citation Patterns in Engineering Doctoral Dissertations. ERIC Document ED243481.

Kushkowski, Jeffrey D., Kathy A. Parsons, and William H. Wiese. 2003. Master's and Doctoral Thesis Citations: Analysis and Trends of a Longitudinal Study. portal: Libraries and the Academy 3 (3):459-479.

Line, Maurice B. 1978. Rank Lists Based on Citations and Library Uses as Indicators of Journal Usage in Individual Libraries. Collection Management 2 (4):313-316.

Musser, Linda R., and Thomas W. Conkling. 1996. Characteristics of Engineering Citations. Science and Technology Libraries 15 (4):41-49.

Patterson, Austin M. 1945. Literature References in Industrial and Engineering Chemistry for 1939. Journal of Chemical Education 22:514-515.

Rieb, Hae-Young. 1993. Citation Analysis: A Case Study of Korean Scientists and Engineers in Electrical and Electronics Engineering. Proceedings of the . . . ASIS Annual Meeting 30:165-171.

Smith, Erin T. 2003. Assessing Collection Usefulness: An Investigation of Library Ownership of the Resources Graduate Students Use. College and Research Libraries:344-355.

Smith, Linda C. 1981. Citation Analysis. Library Trends 30 (1):83-105.

Previous

Contents

Next