422 A Simulated Electronic Availability Study of Serial Articles through a University Library Web Page Thomas E. Nisonger Thomas E. Nisonger is Professor Emeritus in the School of Library and Information Science at Indiana University; e-mail: nisonge@indiana.edu. The author gratefully acknowledges his graduate assistants at the Indiana University School of Library and Information Science, Suzanne S. Switzer and Jennifer A. Brosek, who assisted in a variety of capacities. He also thanks Mary Popp of Indiana University’s Her- man B Wells Library for valuable advice regarding the project and Stephen E. Wiberley, Jr., Professor and Bibliographer for the Social Sciences, University of Illinois at Chicago Library, who provided comparative data gathered at his library. Availability tests as traditional collection evaluation and library perfor- mance measures and their application to electronic resources are ex- plained in this article. A 500-item sample of serial citations was drawn from scholarly journals in 50 disciplines, evenly divided among the sciences, social sciences, humanities, professional fields, and interdisciplinary ar- eas to simulate the needs of Indiana University- Bloomington researchers. It was found that 65.4 percent of the items were electronically available through the library Web page. The electronic availability rate was highest for professional fields, items in the English language, and items published since 2000. The majority of instances of nonavailability occurred because the journal title was not accessible through the Web page. hile the term “availability” can have multiple meanings in both a library and a nonlibrary context, an “availability study” represents a well-established method for evaluation of library collections as well as overall library performance. These studies test whether desired materials (typically books) are shelved in the correct location and can be immediately retrieved by the patron. As will be demonstrated in the following section, more than 90 investi- gations of availability, dating as far back as the 1930s, have been reported in the library and information science literature. A formal “availability study” must be distinguished from other frequent mean- ings of the word in the literature, such as holdings in a library, a journal title’s presence on the Web, or the obtainability of a book in the out-of-print market. Availability studies have been com- pared to systems analysis, as a library may be viewed as a system with the pur- pose of immediately providing desired documents to its clients. An availability study tests how well the system is per- forming that function and can identify specific causes of failure.1 Also, an avail- ability study is considered a measure of user satisfaction, predicated on the assumption that finding a sought-after item results in a happy library patron.2 Indeed, availability studies have occasion- A Simulated Electronic Availability Study 423 ally been termed “satisfaction studies.”3 Stated somewhat differently by Herbert S. White, “the user doesn’t care that the library owns a million books, if he can’t find the one he wants.”4 The overwhelming majority of avail- ability studies have been for known-item searches. In other words, the availability of a specific document or item (usually a book and less frequently a journal article) is tested. A distinction must be made be- tween a “real” and “simulated” approach. The majority of availability investigations would be considered “real” because they were based on self-reports by library pa- trons who were actually trying to locate desired items on the shelves, while a minority have been “simulated” inves- tigations in which the shelf availability of a set of items (presumed to represent user needs) is tested by the investigator. During the 1970s, Paul B. Kantor devel- oped a branching method for analyzing in real studies why a desired item was unavailable. He outlined four branches or barriers to the user not obtaining the desired item: acquisitions (the desired item was not acquired by the library); circulation (all copies are in circulation); library operations (the item is not in the correct shelf location because it is shelved incorrectly, lost, sitting on a book cart, or any of a number of other reasons); and the patron (who cannot locate a correctly shelved item).5 Other branches were used in later modifications of Kantor’s method: the catalog (the item cannot be found in the catalog or the call number is copied incorrectly)6 and the bibliographic (the patron has an incorrect or incomplete citation).7 A number of library and information science authors have advocated an expan- sion of the traditional availability model, which was developed and primarily used for print resources in a single library. In the mid-1990s, Neal K. Kaske called for the creation of “a new valid measure of materials availability” for the “‘virtual’ information environment,” in which pa- trons then used the Internet to check the holdings of multiple libraries via their Online Public Access Catalogs (OPACs) to locate (and request through interlibrary loan) a desired title. Kaske proposed a three-part expansion of the availability concept to include: searching multiple libraries, how soon the item is needed, and the format of the information.8 More recently, Thomas E. Nisonger advocated a redefinition of availability to mean quickly obtaining an item electronically rather than physically on the library shelf. He wrote, “While many patrons may no longer expect the immediate gratifica- tion of finding an item on a library shelf, they may nevertheless expect immediate gratification in locating it electroni- cally.”9 Accordingly, this investigation has adapted the traditional availability con- cept to a Web-based electronic environ- ment. Instead of testing whether a print item is on the shelf, it measures the extent to which the full text of electronic docu- ments can be quickly retrieved on a com- puter screen. Yet, in both print and Web environments, the fundamental research question remains the same: how well is the library performing in providing needed documents to its patrons? Accord- ingly, this article reports an investigation of the availability of 500 serial citations, selected to simulate the needs of research- ers on Indiana University’s Bloomington campus, through the Web page of the Herman B Wells Library. Literature Review This section provides a review (by no means comprehensive) of the literature relating to availability studies in libraries and the use of citations in the evaluation of library collections and electronic re- sources. The earliest known availability study was conducted at the Iowa State College (now renamed Iowa State Uni- versity) Library and described in 1934 by H.V. Gaskill, R.M. Dunbar, and C.H. Brown.10 Since then, according to two literature reviews, more than 90 investiga- tions of availability have been reported. 424 College & Research Libraries September 2009 In 1986, John Mansbridge identified approximately 40 availability studies published from 1934 through the early 1980s.11 Nisonger recently supplemented Mansbridge’s work by reviewing more than 50 investigations of availability (not including any already covered by Man- sbridge) published from 1980 through 2001. Mansbridge indicates that ap- proximately two thirds of the studies he analyzed were based on real users,12 while Nisonger identified 46 real compared to 8 simulated investigations.13 Nisonger’s review identified availabil- ity studies conducted in eleven countries on all continents except South America and Antarctica.14 Examples of book avail- ability analysis include investigations at the University of Zululand in South Africa, based on 353 searches, by Lindiwe E. Zondi;15 and at International Islamic University in Malaysia, based on 441 searches, by Sajjad Ur Rehman, Kokab Arif, and Abdous Sattar Chaudhry.16 More relevant to this article, a number of researchers have investigated the avail- ability of journal articles or included them as part of a multiformat study. Availability data for print journal or serial articles has been reported for: 2,056 searches at the University of North Carolina at Chapel Hill Health Sciences Library by Julia Shaw-Kokot and Claire de la Varre,17 127 searches at Adelphi University by Anne Ciliberti and others,18 an unspeci- fied number of searches at King Fahd University of Petroleum and Minerals in Saudi Arabia by Abdus Sattar Chaudhry and Saleh Ashoor, 19 139 searches at the University of Western Australia by Melanie Harris and Imogen Garner,20 483 searches at the University of New Mexico by Jan Bachmann-Derthick and Sandra Spurlock,21 and 297 searches at the East Tennessee State University College of Medicine by Jennifer Eugenie Roberts.22 To cite illustrative examples of simu- lated studies, Roger Edward Stelk and F.W. Lancaster investigated the shelf availability of two 450-item samples, taken from the shelflist and from a list of recently circulated titles, at the Uni- versity of Illinois at Urbana-Champaign Undergraduate Library.23 Focusing on serials, Chaudhry and Ashoor investi- gated the availability of 100 serial titles from Magazines for Libraries at King Fahd University of Petroleum and Minerals.24 Susan Steynberg and S.F. Rossouw tested the availability of a 307-item sample of journal articles by South African biomedi- cal researchers in the University of Cape Town Medical Library.25 Citations have frequently been used as checklists for collection evaluation for well over 150 years. It should be briefly explained that, in the checklist method, a list of items is checked against the holdings of a library or consortium, as indicated by the catalog (previously the card catalog, now the OPAC). The percentage of items held is considered an indicator of collection strength and quality. For a concise summary of this approach along with an outline of its ad- vantages and disadvantages, see Barbara Lockett’s Guide to the Evaluation of Library Collections.26 In the late 1840s, Charles Coffin Jewett, Assistant Secretary of the Smithsonian Institute, used citations from leading mid-nineteenth century textbooks as a checklist (although he did not use that term) to evaluate the Smithsonian’s library.27 During the 1950s, William L. Emerson drew citations from 23 engi- neering doctoral dissertations to evaluate the Columbia University Engineering Library.28 In the early 1980s, Nisonger checked citations from six political science journals to evaluate the political science collections of Catholic, George Mason, George Washington, Georgetown, and Howard university libraries.29 Reba Lei- ding used citations in 101 undergraduate honors theses written at James Madison University to evaluate the library’s col- lection.30 Many other examples could be listed. Quite recently, Nisonger reviewed studies that used citations from journals, textbooks, dissertations, and theses, as well as faculty publications, as collection evaluation checklists.31 A Simulated Electronic Availability Study 425 In a similar vein, citations have been used as checklists for evaluation of index- ing and/or full-text coverage of electronic databases. For example, David C. Tyler, Signe O. Boudreau, and Susan M. Leach used 6,170 citations from core commu- nication studies journals to evaluate the indexing coverage of three communica- tion studies databases (ComAbstracts, ComIndex, and Communication Abstracts) plus five multidisciplinary databases, including Academic Search Elite and Web of Science.32 Nisonger used 1,166 cita- tions from Library Resources and Technical Services and Collection Building to test the full-text library and information science coverage of the Library Literature and Infor- mation Science Full Text and the EBSCOhost Academic Search Premier databases.33 Similar to this research project, at least three studies have used citations to inves- tigate availability through library Web pages rather than focusing on coverage in specific databases. Thomas Schaffer checked the availability of 368 journal ar- ticle citations from publications by Texas A & M University psychology faculty in 26 electronic databases or resources licensed by that university’s library.34 Jason S. Price analyzed the availability of 861 citations from recent publications by the Department of Biology faculty through the library Web page of the Claremont Colleges.35 In an apparently unpublished paper posted on the Web, Steven J. Squires, Margaret E. Moore, and Susan H. Keesee checked the online avail- ability at the University of North Carolina at Chapel Hill Health Sciences Library of 400 citations from course reading lists and faculty publications plus faculty publica- tions themselves.36 One should briefly mention some other strategies for investigating the availability or accessibility of electronic resources on the Web. Bradley Brazzeal and Amanda Clay Powers investigated the electronic availability of 47 agronomy journals listed in the Journal Citation Reports at 8 ARL libraries.37 Kathleen E. Joswick checked the electronic full-text coverage of 433 core psychology journals (that is, Social Sciences Citation Index source journals) in 8 general databases such as LexisNexis Academic and 4 psychology journal data- bases such as PsyArticles.38 Some studies (for example, Dana M. Caudle and Ce- cilia M. Schmitz, who analyzed the Web sites of 99 ARL libraries) have addressed technical options, such as A to Z lists, OPAC entries, or links to find articles, for accessing electronic resources through the library Web page.39 Finally, while a significant literature exists about the evaluation of electronic database content by methods other than checking citations, the persistence of Web sites or links, citation patterns in numerous disciplines, and print journal subscription holdings, these topics are only tangential to this research project and thus beyond this review’s scope. The Indiana University Bloomington: Campus and Libraries Indiana University was officially estab- lished (under the name State Seminary) in 1820 with classes beginning in Bloom- ington in 1824. The institution was re- named Indiana College in 1828 and then Indiana University in 1838.40 Blooming- ton is now frequently referred to as the “flagship campus” of the eight-campus Indiana University system, with other sites at Fort Wayne, Gary, Indianapolis, New Albany, Kokomo, Richmond, and South Bend. Note that the Indianapolis and Fort Wayne campuses are jointly administered with Purdue, while the Indianapolis campus has a small branch in Columbus, Indiana. In the fall 2007 semester, 38,990 students were enrolled on the Bloomington campus, composed of 30,394 undergraduates, 7,672 graduate, and 924 professional students. There were 1,943 full-time plus 366 part-time faculty as well as 90 professional librarians on the campus.41 There are 332 authorized degree programs on the campus, broken down into 3 technical certificates, 16 nontechnical certificates, 12 associate degrees, 110 baccalaureate degrees, 3 426 College & Research Libraries September 2009 professional degrees, 105 master ’s or specialist degrees, and, most pertinent to this investigation, 83 doctoral degrees.42 The Herman B Wells Library, named after a venerated former Indiana Univer- sity president who, among other things, defended Alfred C. Kinsey’s controversial research about human sexuality, is the central library on the Bloomington cam- pus. Other noteworthy libraries on the campus include the William and Gayle Cook Music Library and the Lilly Library, focusing on rare books and manuscripts. The Library’s OPAC, named IUCAT, pro- vides access to the entire Indiana Univer- sity library system as well as 21 different libraries and 11 Residence Hall Libraries on the Bloomington campus. During the time this research was conducted, the Indiana University Bloomington Librar- ies contained 6.6 million books in 900 languages, ranking 13th in collection size among Association of Research Libraries (ARL) members.43 More germane to this investigation, the IUB library system pro- vided access to 400 electronic databases and 43,000 electronic journals.44 Procedures This simulated study was based on a sample of 500 serial citations, randomly selected from scholarly journals in 50 different subject areas or disciplines. A sample size of 500 was chosen because Kantor recommended a 400- to 500-item sample,45 Blaine H. Hall recommended distribution of 500 survey forms to obtain 400 useable searches for analysis,46 and Mansbridge’s literature review identified 437 as the median sample size.47 The fifty subject areas were chosen af- ter reviewing a list of Indiana University Bloomington’s academic programs, the degrees offered, and student enrollment figures. In almost all cases, graduate- level degrees were offered in the selected areas. The fifty subject areas were equally divided among five broad categories of academic endeavor: the sciences, social sciences, humanities, traditionally inter- disciplinary areas (the word “traditional” is used because many areas of contempo- rary scholarship are now interdisciplin- ary), and professional fields. The source journals for the 50 subjects or disciplines (one journal for each area) may be viewed as a judgment sample. The goal was to select high-quality research- oriented journals that Indiana University faculty and graduate students would be likely to use in their research and seek to access serial articles cited in the journals’ footnotes. Selection priority was given to journals covering their entire field rather than a subdiscipline. The following meth- ods were used to assist journal selection: journal reputation, publisher reputation, whether covered in the Journal Citation Reports (if covered, rank within its subject category by both impact factor and total citations received for each of the three most recent years), Magazines for Librar- ies,48 direct examination of the journal, and examination of Indiana University departmental or program Web pages to help assess departmental research focus. Note that all methods were not used for each journal selected. See Appendix A for the 50 subject areas and the source journal for each. The first 2007 issue was located and 10 citations were randomly selected from at least 4 different articles, except in a few cases where only 3 articles contained citations. The searches were conducted in the researcher ’s office (which has a hardwired connection to the Indiana University Libraries Web page) at various times from approximately 9 A.M. to 6 P.M. Monday through Friday during a period of four and a half months from mid-April through late August of 2007. Searching was based on print copies of the original source articles with the randomly selected citations marked, so there would be no danger of a transcription error of the cita- tion by the researcher. A four-step process, outlined below, was used to search for each randomly selected item: 1. The serial title was entered into the box entitled “Find a specific journal or newspaper online,” accessed by selecting A Simulated Electronic Availability Study 427 “Find Information” on the library Web page’s main menu, and then clicking on “Online Full Text Journals.” This search resulted in a menu (sometimes including false hits) of matches, which listed the years covered, the ISSN, and the database or collection containing the title. All data- bases were searched until the full text was retrieved on the investigator’s computer screen (and reviewed for completeness). If the item was not located in full text, the investigator proceeded to step 2 below; if it was located, the researcher moved to step 3 to check library holdings in print or microform. Note that print and microfor- mat holdings were checked for all items, regardless of electronic availability. 2. The journal title or ISSN plus the specific article’s year, volume, issue, and starting page were entered into the box for a specific article search. Using the SFX by Ex Libris link resolver licensed by the Indiana University Libraries, this strategy would lead directly to the article, if available electronically. This resulted in success in only a few instances, as most electronically available items were retrieved in step 1. A 10-minute limit was placed on searching for the full-text item through the library Web page, predicated on the assumption that, if the document cannot be retrieved within 10 minutes, it is not immediately available. This tactic offers an easily quantifiable objective measure and is modeled upon the ap- proach used in traditional availability studies for print materials. 3. The journal title was searched in the Indiana University OPAC, termed IUCAT, as a final check to verify that an electronic version of the title was not available on the Bloomington campus and to record print or microform holdings for each item in the sample. 4. If a full-text electronic version was not located through the first three steps, a Google search was conducted under the sought-after article’s author or au- thors plus title. In a number of instances, this step revealed that the citation in the source document was incorrect or that an article not accessible through the library Web page was freely available on the Web. The raw results were initially recorded in an MS Word file by the researcher as the searches were conducted. Then an Excel file was created by his graduate assistant to facilitate tabulation by such variables as subject, language, publication date, and the journal’s Library of Congress classification number. The two files were carefully reconciled to ensure that no data errors were introduced. This methodology offers the following advantages: • A reasonable simulation of the re- sources an Indiana University Blooming- ton researcher would likely to be seeking in electronic form through the library Web page Table 1 Full-Text electronic availability of Simulated Sample Category Number available Percentage Not available Percentage Area Studies 100 51 51% 49 49% Humanities 100 45 45% 55 55% Professional Fields 100 81 81% 19 19% Sciences 100 79 79% 21 21% Social Sciences 100 71 71% 29 29% Total 500 327 65.4% 173 34.6% 428 College & Research Libraries September 2009 • The potential bias of using citations from faculty publications or course syl- labi (that is, available items may be more likely to be cited) is overcome49 • A focus is on the collection evalua- tion issue of whether the item is available on the Bloomington campus through the Indiana University Libraries Web page • The random selection method results in interdisciplinary citations, reflecting the nature of contemporary scholarship. Results Table 1, organized by five broad catego- ries, presents this investigation’s basic results. Note that this categorization is based on the source journals, not the actual subject of the sought-after items, which often varied due to interdisciplin- ary citations. In total, 65.4 percent of the items were available (based on the criteria outlined above) through the Indiana Uni- versity Web page. Wide variation in the availability rates among source journal category is evident. The two strongest categories were professional fields (81%) and the sciences (79%), followed by the social sciences (71%). A far lower avail- ability rate (51%) can be observed for area studies/interdisciplinary areas (which will subsequently be referred to as “area studies”). The weakest category was the humanities, in which less than half the cited items were available electronically (45%). In addition, 18 items unavailable through the library Web page were ob- tained in full text through a follow-up Google search. If these were to be count- ed, the overall availability rate would in- crease to 69 percent and the rates by broad category would rise to 86 percent for the sciences, 84 percent for professional fields, 75 percent for the social sciences, 54 per- cent for area studies, and 46 percent for the humanities. However, it is uncertain how many library users would conduct follow-up Google searches and ques- tionable whether the library should be credited for items located through Google (although its Web page does offer a link to Google) as they were not licensed by the library. Thus, the remaining analysis in this article will focus on the 65.4 percent of items available directly through the library Web page. Why the variation among categories? A supposition that the humanities and area studies were the two weakest categories simply because the Indiana University Bloomington Libraries attach a low col- lecting priority to them is not a reason- able explanation because the library supports the campus’s strong tradition of emphasizing both the humanities and area studies. As will be elaborated upon below, the language and age composition of the samples for each category partially explains the variation. Also, drawing upon Everett M. Rogers’ diffusion of innovations theory,50 one is tempted to speculate that the speed at which an area is perceived to adapt to technology (the scientific areas as “early adopters” and the humanities as “laggards”) influences vendor marketing of electronic resources. Table 2 tabulates the explanations for nonavailability for the 173 items not immediately available through the Indi- ana University Libraries Web page. The most frequent explanation, accounting for more than three fifths (62.4%) of the nonavailable items, was the fact that the title containing the sought-after citation was not electronically available (in other words, was not accessible through the library’s Web page, either through licens- ing or a link to an open-access source). Indeed, nonavailability of the title itself was the most frequent reason for non- availability in all five broad categories, ranging from 77.6 percent in area studies to 38.1 percent in the sciences. Next, in nearly a quarter of the instances of non- availability (23.1%), the title was actually available, but the citation was earlier than the available run. Indicating a consistent pattern across knowledge domains, this was the second most frequent explanation in each of the broad categories, ranging from 44.8 percent in social sciences to 8.2 A Simulated Electronic Availability Study 429 Table 2 Reasons for Nonavailability Reason Number of Unavailable Items Percentage area Studies 49 49%* Title not available electronically 38 77.6%** Citation earlier than available run 4 8.2% Combination of factors 2 4.1% Title available, but coverage not complete 2 4.1% Citation later than available run 1 2% Title available, but item not retrieved 1 2% Title available, but wrong citation given 1 2% Humanities 55 55% Title not available electronically 40 72.7% Citation earlier than available run 10 18.2% Title available, but coverage not complete 3 5.5% Citation later than available run 2 3.6% Professional Fields 19 19% Title not available electronically 8 42.1% Citation earlier than available run 6 31.6% Title available, but wrong citation given 2 10.5% Citation later than available run 1 5.3% Title available, but coverage not complete 1 5.3% Title available, but item not retrieved 1 5.3% Sciences 21 21% Title not available electronically 8 38.1% Citation earlier than available run 7 33.3% Citation later than available run 2 9.5% Title available, but item required more than 10 minutes to retrieve 1 4.8% Title available, but item not retrieved 1 4.8% Title available, but wrong citation given 1 4.8% Title available, but searching error 1 4.8% Social Sciences 29 29% Title not available electronically 14 48.3% Citation earlier than available run 13 44.8% Title available, but item not retrieved 1 3.4% Combination of factors 1 3.4% Grand Total 173 34.6% Title not available electronically 108 62.4% 430 College & Research Libraries September 2009 percent in area studies. Tied for a distant third place (although not consistently third among the five categories) and each accounting for 3.5 percent of availability failures, were a citation more current than the available run and incomplete coverage for an available title—defined as cases in which the volume and issue were avail- able without including the specific item. It is significant that almost 90 percent of nonavailable items could be attributed to the fact that the needed volume was not accessible through the Web page. The remaining causes for electronic nonavailability were scattered among a variety of situations. Four failures (2.3%) were due to an error in the original cita- tion (remember that the search was based on a copy of the source article rather than a transcription of the citation by the investigator). The error was discovered through the follow-up Google search, and then the correct citation was verified to be electronically available through the library Web page. In 4 instances (2.3% of failures), a theoretically available item was not accessible due to a retrieval er- ror; in other words, clicking on an article or volume number would not lead to its display on the screen. Three failures (1.7%) were caused by a combination of factors, such as an incorrect citation in combination with a faulty search strat- egy by the investigator. Finally, in one instance, the cited item was retrieved after the 10-minute limit had lapsed; and, in another, a searcher error (this investiga- tor’s) was responsible for not finding an electronically retrievable article due to an ineffective searching strategy (both representing 0.6% of failures). How can Kantor’s branching method, which identifies (for a print environment) barriers to finding the desired item, be applied to the results of this study? The “bibliographical branch” (beginning the search with an incorrect citation) ac- counted for 2.3 percent (4 of 173) of the retrieval failures because of a citation er- ror in the source journal. Note that there were numerous other cases of source journal citation error that did not result in failed searches. The “acquisitions branch” (the desired item was not acquired), for which the electronic equivalent would be not licensing or linking to the item, represented the major barrier to electronic availability, causing 160 of the 173 (92.5%) instances of nonavailability. The “circu- lation branch” (the sought-after item is in use) would usually not apply in the electronic world (except for netLibrary’s one-book, one-user policy or a maximum number of simultaneously licensed users) Table 2 Reasons for Nonavailability Reason Number of Unavailable Items Percentage Citation earlier than available run 40 23.1% Citation later than available run 6 3.5% Title available, but coverage not complete 6 3.5% Title available, but item not retrieved 4 2.3% Title available, but wrong citation given 4 2.3% Combination of factors 3 1.7% Title available, but item required more than 10 minutes to retrieve 1 0.6% Title available, but searching error 1 0.6% *Percentage of sample that was unavailable **Percentage of unavailable items A Simulated Electronic Availability Study 431 and did not lead to any of this investiga- tion’s failures. The effectiveness of the Web page’s electronic article retrieval devices (described in the “Procedures” section) would arguably approximate Kantor’s “catalog branch” (can the item be located in the catalog?). Only one failure (0.6% the 173 total)—the instance in which more than 10 minutes was required for retrieval because access was not provided under the exact title—can be assigned to this branch. Library Operations (the item is not shelved correctly) does not have an obvious equivalent because electronic items are not shelved. The “patron” or “user” branch (a correctly shelved item cannot be located) does have an obvious electronic equivalent—the user’s ability to retrieve an item—but was not examined here since this was a simulated study. However, the one instance of nonretrieval due to the investigator’s searching error (0.6% of failures) could be viewed as a patron failure, although the percentage would presumably be higher in an inves- tigation based on actual users. The four cases of failure to retrieve a theoretically available citation (2.3%) (specifically the needed volume number could not be pulled up) appear to have been the fault of an external site to which the library linked and therefore do not readily fit into Kan- tor’s framework, although they might be considered a type of “operations” failure. Finally, 1.7 percent (3) of the failures were due to a combination of reasons and thus cannot be attributed to a single branch. An analysis of availability by language is illustrated in Table 3. It is striking that 72.5 percent (321 of 443) of the English language citations were available through the Library Web page, contrasted to only 10.5 percent (6 of 57) for citations in other languages. The science, profes- Table 3 analysis of electronic availability by language Category Number available Not available Number Percent Number Percent area Studies English 68 48 70.6% 20 29.4% Non-English 32 3 9.4% 29 90.6% Humanities English 81 44 54.3% 37 45.7% Non-English 19 1 5.3% 18 94.7% Professional Fields English 98 80 81.6% 18 18.4% Non-English 2 1 50% 1 50% Sciences English 98 78 79.6% 20 20.4% Non-English 2 1 50% 1 50% Social Sciences English 98 71 72.4% 27 27.6% Non-English 2 0 0% 2 100% Grand Total English 443 321 72.5% 122 27.5% Non-English 57 6 10.5% 51 89.5% 432 College & Research Libraries September 2009 Table 4 analysis of electronic availability by Publication Date and Category Time Period* Number of Items available Not available Number Percent Number Percent Number Percent area Studies Before 1900 6 6% 2 33.3% 4 66.7% 1920s 2 2% 0 0% 2 100% 1930s 6 6% 3 50% 3 50% 1940s 2 2% 1 50% 1 50% 1950s 4 4% 2 50% 2 50% 1970s 8 8% 2 25% 6 75% 1980s 12 12% 8 66.7% 4 33.3% 1990s 24 24% 12 50% 12 50% 2000s 36 36% 21 58.3% 15 41.7% Total 100 100% 51 51% 49 49% Humanities Before 1900 2 2% 0 0% 2 100% 1900-1909 2 2% 2 100% 0 0% 1920s 1 1% 0 0% 1 100% 1930s 4 4% 2 50% 2 50% 1950s 5 5% 2 40% 3 60% 1960s 2 2% 0 0% 2 100% 1970s 12 12% 6 50% 6 50% 1980s 18 18% 8 44.4% 10 55.6% 1990s 24 24% 10 41.7% 14 58.3% 2000s 30 30% 15 50% 15 50% Total 100 100% 45 45% 55 55% Professional Fields 1960s 1 1% 1 100% 0 0% 1970s 2 2% 1 50% 1 50% 1980s 5 5% 4 80% 1 20% 1990s 39 39% 34 87.2% 5 12.8% 2000s 53 53% 41 77.4% 12 22.6% Total 100 100% 81 81% 19 19% Sciences 1930s 2 2% 1 50% 1 50% 1960s 1 1% 1 100% 0 0% 1970s 5 5% 4 80% 1 20% 1980s 10 10% 6 60% 4 40% 1990s 29 29% 24 82.8% 5 17.2% A Simulated Electronic Availability Study 433 sional field, and social science citations were almost exclusively in English, while approximately one fifth of the humani- ties citations and one third of those in area studies were in languages other than English. The two categories with the highest proportion of non-English citations, humanities and area studies, displayed the lowest overall availability rates, but their language composition is only a partial explanation for the weaker performance. Although not indicated in table 3, eleven languages besides English were included in the 500-item sample. None of the following were available: 1 Dutch, 1 Hebrew, 1 Turkish, 3 Catalan, 3 Italian, and 4 Chinese citations. One of 2 Portuguese, 4 Spanish, 8 Russian, and 11 German citations were available. Finally, 2 of 19 French citations were available. Table 4 summarizes availability by publication date and category. The “Grand Total” section clearly shows a positive association between citation cur- Table 4 analysis of electronic availability by Publication Date and Category Time Period* Number of Items available Not available Number Percent Number Percent Number Percent 2000s 53 53% 43 81.1% 10 18.9% Total 100 100% 79 79% 21 21% Social Sciences Before 1900 2 2% 0 0% 2 100% 1900-1909 1 1% 0 0% 1 100% 1920s 2 2% 0 0% 2 100% 1960s 2 2% 1 50% 1 50% 1970s 4 4% 2 50% 2 50% 1980s 8 8% 3 37.5% 5 62.5% 1990s 44 44% 31 70.5% 13 29.5% 2000s 37 37% 34 91.9% 3 8.1% Total 100 100% 71 71% 29 29% Grand Total Before 1900 10 2% 2 20% 8 80% 1900-1909 3 .6% 2 66.7% 1 33.3% 1920s 5 1% 0 0% 5 100% 1930s 12 2.4% 6 50% 6 50% 1940s 2 .4% 1 50% 1 50% 1950s 9 1.8% 4 44.4% 5 55.6% 1960s 6 1.2% 3 50% 3 50% 1970s 31 6.2% 15 48.4% 16 51.6% 1980s 53 10.6% 29 54.7% 24 45.3% 1990s 160 32% 111 69.4% 49 30.6% 2000s 209 41.8% 154 73.7% 55 26.3% Total 500 100% 327 65.4% 173 34.6% *If no items were published during a particular decade, that decade has been omitted from the table. 434 College & Research Libraries September 2009 rency and availability through the library Web page. A linear relationship in which the availability rate declines with age is evident for the four most recent decades: 73.7 percent of items published during the 2000s; 69.4 percent of those published during the 1990s; 54.7 percent of citations from the 1980s; and 48.4 percent of those from the 1970s were available in full text. In contrast, only 20 percent of the items published before 1900 were available. Also, as would be expected, categories with more current citations tended to have higher availability rates. While 41.8 percent of all the citations were published in the 2000s, the figure was 53 percent in the two categories with the highest overall availability rates in table 1— professional fields (81%) and the sciences (79%). The percentage published in the 2000s did not even reach 40 percent in any other cat- egory: social sciences (37%), area studies (36%), and the humanities (30%). Table 5 analyzes the simulated sample according to the Library of Congress class number for the cited journal, as indicated in WorldCat. Remember that the five broad categories analyzed in this paper are based on the source journals, not the cited journals. Three of the cited titles were not found in WorldCat, while 11 of those found in WorldCat (counted in the table as “Other”) used a different classification system, such as the National Library of Medicine, or otherwise lacked an LC classification number. Although ancillary to the primary research question, table 5 illustrates the interdisciplinary nature of contemporary scholarship—a well-known phenomenon supported by hundreds of citation stud- ies. One can see that 16 LC classes are represented in area studies with the most prevalent, D (old world history), account- ing for only 25 percent of the total. A simi- lar pattern appears for the humanities (10 LC classes, with the largest accounting for only 25%), professional fields (11 classes, with the largest producing only 27%), and the social sciences (15 LC classes, with the largest contributing only 27%). In contrast, the sciences displayed low interdisciplinarity (only four classes were represented) and high concentration in one class (Q—the class for science itself), which accounted for 80 percent of the cita- tions in science source journals. Overall, 19 of 21 LC classes were included in the total 500-item sample, with U (Military Science) and V (Naval Science) the two exceptions. Discounting S (Agriculture), in which there were only two items, the highest electronic availability rate—90 percent— was found in classes E (U.S. History) and K (Law). The availability rate exceeded 80 percent in three other LC classes: M (Music and Books on Music)—85.7%; Z (Bibliography, Library Science, and Gen- eral Information Resources)—84.6%; and J (Political Science)—83.9%, although there were only seven citations in M. Three other classes had availability rates above 70 percent: H (Social Sciences)—79.5%; Q (Science)—74.2%; and L (Educa - tion)—71.4% (although the rate for L, as with M, is based on a less than robust da- taset). Two traditional humanities classes that were well represented in the sample had availability rates considerably below the 65.4 percent total figure: D (History of Europe, Asia, Africa, Australia, New Zea- land, etc.) at 42.5 percent and P (Language and Literature) at 46.8 percent. While this investigation focused on electronic availability through the library Web page, holdings on the Bloomington campus in print and microform were also analyzed and the findings summarized in table 6. It is noteworthy that a somewhat larger proportion of the sample (73%) was held in print than was available electroni- cally (65.4%). Items available electroni- cally were more likely to be held in print than unavailable items (83.5% contrasted to 53.2%), suggesting that the latter group may have contained more marginal mate- rial. A similar pattern can be observed for microform (combining film and fiche) in which 7.3 percent of available items but only 4.6 percent of nonavailable ones were held in the format. Note that 16 items A Simulated Electronic Availability Study 435 Table 5 analysis of electronic availability by library of Congress Classification Number Category Number of Items available Not available Number Percent Number Percent Number Percent area Studies A 5 5% 4 80% 1 20% B 5 5% 2 40% 3 60% C 2 2% 0 0% 2 100% D 25 25% 12 48% 13 52% E 6 6% 6 100% 0 0% F 2 2% 1 50% 1 50% G 3 3% 2 66.7% 1 33.3% H 17 17% 11 64.7% 6 35.3% J 5 5% 3 60% 2 40% K 1 1% 1 100% 0 0% L 2 2% 1 50% 1 50% N 1 1% 1 100% 0 0% P 7 7% 2 28.6% 5 71.4% Q 6 6% 1 16.7% 5 83.3% R 1 1% 1 100% 0 0% S 1 1% 1 100% 0 0% Newspaper 9 9% 2 22.2% 7 77.8% Not Found 1 1% 0 0% 1 100% Other 1 1% 0 0% 1 100% Total 100 100% 51 51% 49 49% Humanities A 11 11% 4 36.4% 7 63.6% B 17 17% 10 58.8% 7 41.2% C 1 1% 0 0% 1 100% D 9 9% 1 11.1% 8 88.9% E 1 1% 0 0% 1 100% G 12 12% 6 50% 6 50% H 2 2% 1 50% 1 50% M 7 7% 6 85.7% 1 14.3% N 9 9% 1 11.1% 8 88.9% P 25 25% 12 48% 13 52% Newspaper 4 4% 4 100% 0 0% Not Found 1 1% 0 0% 1 100% Other 1 1% 0 0% 1 100% 436 College & Research Libraries September 2009 Table 5 analysis of electronic availability by library of Congress Classification Number Category Number of Items available Not available Number Percent Number Percent Number Percent Total 100 100% 45 45% 55 55% Professional Fields A 1 1% 0 0% 1 100% B 4 4% 4 100% 0 0% D 2 2% 2 100% 0 0% H 27 27% 24 88.9% 3 11.1% J 10 10% 8 80% 2 20% K 8 8% 7 87.5% 1 12.5% L 5 5% 4 80% 1 20% P 2 2% 2 100% 0 0% Q 5 5% 5 100% 0 0% R 19 19% 13 68.4% 6 31.6% Z 10 10% 8 80% 2 20% Newspaper 1 1% 1 100% 0 0% Other 6 6% 3 50% 3 50% Total 100 100% 81 81% 19 19% Sciences Q 80 80% 63 78.8% 17 21.2% R 3 3% 3 100% 0 0% T 11 11% 7 63.6% 4 36.4% Z 3 3% 3 100% 0 0% Not Found 1 1% 1 100% 0 0% Other 2 2% 2 100% 0 0% Total 100 100% 79 79% 21 21% Social Sciences A 1 1% 1 100% 0 0% B 5 5% 5 100% 0 0% D 4 4% 2 50% 2 50% E 3 3% 3 100% 0 0% F 1 1% 1 100% 0 0% G 6 6% 3 50% 3 50% H 27 27% 22 81.5% 5 18.5% J 16 16% 15 93.8% 1 6.2% K 1 1% 1 100% 0 0% N 1 1% 1 100% 0 0% A Simulated Electronic Availability Study 437 Table 5 analysis of electronic availability by library of Congress Classification Number Category Number of Items available Not available Number Percent Number Percent Number Percent P 13 13% 6 46.2% 7 53.8% Q 6 6% 3 50% 3 50% R 11 11% 5 45.5% 6 54.5% S 1 1% 1 100% 0 0% T 1 1% 1 100% 0 0% Newspaper 2 2% 1 50% 1 50% Other 1 1% 0 0% 1 100% Total 100 100% 71 71% 29 29% Grand Total A 18 3.6% 9 50% 9 50% B 31 6.2% 21 67.7% 10 32.3% C 3 .6% 0 0% 3 100% D 40 8% 17 42.5% 23 57.5% E 10 2% 9 90% 1 10% F 3 .6% 2 66.7% 1 33.3% G 21 4.2% 11 52.4% 10 47.6% H 73 14.6% 58 79.5% 15 20.5% J 31 6.2% 26 83.9% 5 16.1% K 10 2% 9 90% 1 10% L 7 1.4% 5 71.4% 2 28.6% M 7 1.4% 6 85.7% 1 14.3% N 11 2.2% 3 27.3% 8 72.7% P 47 9.4% 22 46.8% 25 53.2% Q 97 19.4% 72 74.2% 25 25.8% R 34 6.8% 22 64.7% 12 35.3% S 2 .4% 2 100% 0 0% T 12 2.4% 8 66.7% 4 33.3% Z 13 2.6% 11 84.6% 2 15.4% Newspaper 16 3.2% 8 50% 8 50% Not Found 3 .6% 1 33.3% 2 66.7% Other 11 2.2% 5 45.5% 6 54.5% Total 500 100% 327 65.4% 173 34.6% 438 College & Research Libraries September 2009 Table 6 analysis by Format Category Number Held in Print Held in Microformat Held in Print or Microformat Held in any Format No. % No. % No. % No. % area Studies Available Electronically 51 45 88.2% 5 9.8% 47 92.2% 51 100% Not Available Electronically 49 21 42.9% 4 8.2% 25 51% 25 51% Total 100 66 66% 9 9% 72 72% 76 76% Humanities Available Electronically 45 32 71.1% 5 11.1% 35 77.8% 45 100% Not Available Electronically 55 31 56.4% 2 3.6% 33 60% 33 60% Total 100 63 63% 7 7% 68 68% 78 78% Professional Fields Available Electronically 81 68 84% 5 6.2% 69 85.2% 81 100% Not Available Electronically 19 7 36.8% 1 5.3% 8 42.1% 8 42.1% Total 100 75 75% 6 6% 77 77% 89 89% Sciences Available Electronically 79 67 84.8% 2 2.5% 67 84.8% 79 100% Not Available Electronically 21 13 61.9% 0 0% 13 61.9% 13 61.9% Total 100 80 80% 2 2% 80 80% 92 92% Social Sciences Available Electronically 71 61 85.9% 7 9.9% 63 88.7% 71 100% Not Available Electronically 29 20 69% 1 3.4% 21 72.4% 21 72.4% Total 100 81 81% 8 8% 84 84% 92 92% Grand Total Available Electronically 327 273 83.5% 24 7.3% 281 85.9% 327 100% Not Available Electronically 173 92 53.2% 8 4.6% 100 57.8% 100 57.8% Total 500 365 73% 32 6.4% 381 76.2% 427 85.4% A Simulated Electronic Availability Study 439 were available and held in three different formats: print, microform, and electronic. Finally, because 100 of the electronically unavailable items (57.8%) were held in either print or microform, 427 of the 500- item sample (85.4%) should have been obtainable on the Bloomington campus. Although unsurprising, this finding does demonstrate that electronic and traditional (that is, print and microform) resources are stronger in combination than either one by itself. Of the 92 nonavailable items held in print, 21 (22.8%) items were in remote storage (the Auxiliary Library Facility or ALF) and definitely would not have been immediately available to patrons. (One must request an item before noon to receive it by 5 P.M.51) Note that 91 (33.3%) of the 273 items both available electronically and held in print were in remote storage, although it is question- able how often patrons would seek the print version of an electronically available document. In total, 30.7 percent (112 of 365) of the print-held items were in the ALF and thus not immediately available. No attempt was made to determine the actual shelf availability of the remaining print or microform items. While these formats usually do not circulate in the IU- Bloomington library system, the required volume or issue could have been in use by a patron or misshelved. Comparison of Results with Other Availability Studies After data from an evaluation project are tabulated and analyzed, a fundamental is- sue remains: interpretation of the results. What score should one expect? Although no other investigation of availability found in the literature review is identical to this one, a review of the results from other availability studies provides some context for assessing the results found here. Compared to real (that is, patron based), known-item availability studies for mostly print resources, this investiga- tion’s 65.4 percent electronic availability rate is somewhat higher than the average rates found by both Mansbridge (61% for 26 studies)52 and Nisonger (a 61.3% unweighted average or a 63.1% weighted average for 46 investigations).53 More spe- cifically, 65.4 percent availability is higher than 18 of Mansbridge’s data points, lower than 7 and essentially tied with one. Un- fortunately, Mansbridge does not report the average availability rate or specific results from simulated investigations.54 In the Nisonger literature review, the avail- ability rates ranged from 33.8 percent to 83.8 percent in user-based studies. This article’s results were higher than 27 of these investigations, but lower than 19. Nisonger also found that the unweighted mean availability rate for 8 simulated investigations was 61.8 percent and the weighted mean 60.6 percent, with the specific results ranging from 13.0 percent to 84.0 percent. If the 13.0 percent score, which could be considered an outlier, is disregarded, availability rises to an unweighted mean of 68.8 percent and a weighted mean of 69.1 percent. Five of the eight simulated studies in Nisonger’s review resulted in higher availability rates than the one found here,55 including a 69.4 percent rate in the University of Cape Town Medical Library for journal articles published by South African biomedical researchers.56 Compared to availability rates for print journal or serial articles in real studies, these results were considerably lower than the 80.9 percent rate found at the University of North Carolina Chapel Hill Health Sciences Library,57 but higher than 58.6 percent at King Fahd University of Petroleum and Minerals,58 56 percent at the University of New Mexico,59 55 per- cent at East Tennessee State University,60 54 percent at the University of Western Australia,61 and 44.9 percent at Adelphi University.62 More direct comparisons can be made with studies investigating the electronic availability of journal citations through library Web pages. Schaffer discovered that “less than one-third” of the journal articles cited by Texas A & M psychol- 440 College & Research Libraries September 2009 ogy faculty were available in full text in at least one of 26 electronic databases licensed by the university library63— equally approximately half the 65.4 per- cent rate found here and even less than the 71 percent availability rate for the so- cial sciences. In contrast, the 81 percent electronic availability rate for journal ar- ticles cited by Claremont Colleges biol- ogy faculty in Price’s investigation64 and the 78 percent rate reported by Squires, Moore, and Keesee for the health sci- ences at the University of North Carolina at Chapel Hill65 are remarkably close to the 79 percent rate for the sciences and 81 percent for professional field (health sciences can be legitimately compared to both the sciences and professional fields) found here. Preliminary data from an unpublished investigation of the electronic availability of citations from social science faculty publications at the University of Illinois at Chicago by Stephen E. Wiberley, Jr. show a 78.4 percent rate (69 of 88) for journals, some- what higher than this study’s result for the social sciences. Note that Wiberley’s overall availability rate was 45.3 percent (72 of 159) when all formats are consid- ered.66 In summary, this investigation’s overall availability rate is somewhat higher than for most studies of print materials, but not enough comparable studies of electronic availability were identified to allow generalizations. In fact, no other identified study was pre- cisely comparable with this one. Limitations to the Study The fact that this was a simulated study that did not incorporate real users is undoubtedly the biggest limitation. A number of other limitations must be ac- knowledged. The investigator searched in his office (which is hardwired for high-speed access) rather than on his home computer, where response time is slower; it is possible that Indiana Uni- versity users searching from a remote location without high-speed Internet ac- cess would obtain poorer results because some items were not retrieved within the 10-minute limit. The focus is on research rather than teaching needs. Since it is a simulated study, the results indicate potential rather than actual availability. Formats other than serials have not been included in the simulated methodology. A different set of sample parameters might have produced somewhat dif- ferent findings. Since the data were gathered, the Web page for the Indiana University Bloomington libraries has been redesigned and it is unknown how or if this impacts availability. Conclusions This research project demonstrates once again that traditional collection evaluation or library performance measures devel- oped for a print environment can be modi- fied for application to electronic resources or a hybrid library. More specifically, the availability study method has been adapt- ed to test near-immediate full-text retrieval on a terminal screen rather than location of an item on a library shelf, while the follow- up analysis of print holdings incorporates the approach of a citation-based checklist. This investigation differs from the other electronic availability studies reviewed here because: it is based on a simulated sample rather than citations from library clients; 50 subject areas rather than one serve as the source for the citations; and the results are compared with those from availability studies for print resources. The investigation’s conclusions may be summarized as follows: • the simulated electronic availabil- ity rate for serial citations appears to be slightly higher than the overall average rates found in patron-based availability studies for print materials and for most patron-based investigations of print serial availability; • as the literature review did not lo- cate a single electronic availability study that incorporated actual searching by pa- trons, the results that would be obtained by a “real” electronic availability study are unknown; A Simulated Electronic Availability Study 441 • the so-called “hybrid library,” combining electronic and print resources, meets a larger proportion of simulated user need than would an all-electronic or an all-print collection—at least as of mid 2007; • electronic availability varies widely according to the sought-after item’s sub- ject, age, and language; • as the continuing need (as indicated by citations in scholarly literature) for old- er and non-English material is confirmed, research libraries should either expand their licensing for these parameters (if provided by vendors) or maintain print collections; • modification of Kantor’s branching method for an electronic environment indicates that the “acquisitions bar- rier”—the needed item was not licensed or linked to—is the primary cause of nonavailability. Strategies for future research investi- gating electronic availability include: • a “real” study incorporating actual users (for which a lower availability rate might be expected due to user error); • a thorough investigation of third- party failure as a barrier to availability; • implementation of the identi- cal simulated methodology at future yearly intervals on the Indiana University Bloomington library Web page for longi- tudinal comparison; • implementation of a similar simu- lated methodology on the Web pages of other large research libraries for cross- sectional comparison; • use of a simulated investigation, based on a sample created to represent teaching rather than research needs; • use of log transaction analysis; and • various permutations on the strate- gies outlined above. Notes 1. Lynn B. Chandler, “Book Availability in the Cameron Village Regional Library,” (Master’s paper, University of North Carolina at Chapel Hill, 1998), 2. 2. F.W. Lancaster, If You Want to Evaluate Your Library. . . (Champaign, Ill.: University of Il- linois, Graduate School of Library and Information Science, 1993), 12, who cites Ranganathan’s Second Law of Library Science, “Every Reader His Book.” 3. Gene E. Rinkel and Patricia McCandless, “Application of a Methodology Analyzing User Frustration,” College & Research Libraries 44 (Jan. 1983): 29–37. 4. This is an approximate quotation, based on the author’s memory, from Herbert S. White’s presentation at the Indiana University School of Library and Information Science on May 9, 2003. 5. Paul B. Kantor, “Availability Analysis,” Journal of the American Society for Information Science 27 (Sept.–Oct. 1976): 311–19. 6. Judith B.Wood, Julius J. Bremer, and Susan A. Saraidaridis, “Measurement of Service at a Public Library,” Public Library Quarterly 2 (summer 1980): 49–57. 7. Faraj Mohamed Kuraim, “The Principal Factors Causing Reader Frustration in a Public Library” (Ph.D. diss., Case Western Reserve University, 1983). 8. Neal K. Kaske, “Materials Availability Model and the Internet,” Journal of Academic Librari- anship 20 (Nov. 1994): 317–18. 9. Thomas E. Nisonger, “A Review and Analysis of Library Availability Studies,” Library Resources & Technical Services 51 (Jan. 2007): 36. 10. H.V. Gaskill, R.M. Dunbar, and C.H. Brown, “An Analytical Study of the Use of a College Library,” Library Quarterly 4 (Oct. 1934): 564–87. 11. John Mansbridge, “Availability Studies in Libraries,” Library & Information Science Research 8 (Oct.–Dec. 1986): 299–314. 12. Ibid., 300. 13. Nisonger, “A Review and Analysis of Library Availability Studies,” 33. 14. Ibid. 15. Lindiwe E. Zondi, “Measuring Availability and Non-availability Rates at the University of Zululand Library,” South African Journal of Library & Information Science 64 (June 1996): 108–13. 16. Sajjad Ur Rehman, Kokab Arif, and Abdous Sattar Chaudhry, “Availability Analysis: Conduct, Comparison and Applications,” Australian Academic & Research Libraries 25 (Mar. 1994): 19–26. 442 College & Research Libraries September 2009 17. Julia Shaw-Kokot and Claire de la Varre, “Using a Journal Availability Study to Improve Access,” Bulletin of the Medical Library Association 89 (Jan. 2001): 21–28. 18. Anne Ciliberti et al., “Empty Handed? A Material Availability Study and Transaction Log Analysis Verification,” Journal of Academic Librarianship 24 (July 1998): 284. 19. Abdus Sattar Chaudhry and Saleh Ashoor, “Comprehensive Materials Availability Studies in Academic Libraries,” Journal of Academic Librarianship 20 (Nov. 1994): 300–05. 20. Melanie Harris and Imogen Garner, “Using an Availability Survey to Improve Service at a University Library,” Australian Academic & Research Libraries 23 (Mar. 1992): 25–34. 21. Jan Bachmann-Derthick and Sandra Spurlock, “Journal Availability at the University of New Mexico,” in Advances in Serials Management, vol. 3, eds. Jean G. Cook and Marcia Tuttle (Greenwich, Conn.: JAI Press, 1989), 173–212. 22. Jennifer Eugenie Roberts, “Journal Availability Study” (Master’s paper, University of North Carolina at Chapel Hill, 1989). 23. Roger Edward Stelk and F.W. Lancaster, “The Use of Shelflist Samples in Studies of Book Availability,” Collection Management 13, no. 4 (1990): 19–24. 24. Chaudhry and Ashoor, “Comprehensive Materials Availability Studies in Academic Librar- ies.” 25. Susan Steynberg and S.F. Rossouw, “The Availability of Research Journals in South African Academic Medical Libraries,” South African Medical Journal 83 (Nov. 1993): 837–39. 26. Guide to the Evaluation of Library Collections, ed. Barbara Lockett (Chicago: American Library Association, 1989), 5–6. 27. C.C. Jewett, “Report of the Assistant Secretary Relative to the Library, Presented December 13, 1848,” in Third Annual Report of the Board of Regents of the Smithsonian Institution to the Senate and House of Representatives (Washington, D.C.: Tippin and Streeper, 1849): 39–47. 28. William L. Emerson, “Adequacy of Engineering Resources for Doctoral Research in a University Library,” College & Research Libraries 18 (Nov. 1957): 455–60, 504. 29. Thomas E. Nisonger, “A Test of Two Citation Checking Techniques for Evaluating Political Science Collections in University Libraries,” Library Resources & Technical Services 27 (Apr./June 1983): 163–76. 30. Reba Leiding, “Using Citation Checking of Undergraduate Honors Thesis Bibliographies to Evaluate Library Collections,” College & Research Libraries 66 (Sept. 2005): 417–29. 31. Thomas E. Nisonger, “Use of the Checklist Method for Content Evaluation of Full Text Databases: An Investigation of Two Databases Based on Citations from Two Journals,” Library Resources & Technical Services 52 (Jan. 2008): 4–17. 32. David C. Tyler, Signe O. Boudreau, and Susan M. Leach, “The Communication Studies Researcher and the Communication Studies Indexes,” Behavioral & Social Sciences Librarian 23, no. 2 (2005): 119–46. 33. Nisonger, “Use of the Checklist Method for Content Evaluation of Full Text Databases.” 34. Thomas Schaffer, “Psychology Citations Revisited: Behavioral Research in the Age of Electronic Resources,” Journal of Academic Librarianship 30 (Sept. 2004): 354–60. 35. Jason S. Price, “How Many Journals Do We Have? An Alternative Approach to Journal Collection Evaluation Through Local Cited Article Analysis,” Serials 20 (July 2007): 134–41. 36. Steven J. Squires, Margaret E. Moore, and Susan H. Keesee, “Electronic Journal Availability Study.” Available online at www.eblip4.unc.edu/papers/Squires.pdf. [Accessed 23 May 2008]. 37. Bradley Brazzeal and Amanda Clay Powers, “Electronic Access to Agricultural Journals: An Agronomy Case Study,” Serials Review (Sept. 2007): 155–60. 38. Kathleen E. Joswick, “Full Text Psychology Journals Available from Popular Library Da- tabases,” Journal of Academic Librarianship 32 (2006): 349–54. 39. Dana M. Caudle and Cecilia M. Schmitz, “Web Access to Electronic Journals with Databases in ARL Libraries,” Journal of Web Librarianship 1, no. 1 (2007): 3–26. 40. Wikipedia, “Indiana University (Bloomington).” Available online at http://en.wikipedia. org/wiki/Indiana_University_%28Bloomingto. [Accessed 29 November 2007]. 41. Indiana University, Indiana University Fact Book 2007–08 (Bloomington, Ind.: Indiana Uni- versity, University Reporting and Research, 2008), 2. Available online at http://factbook.indiana. edu/fbook07/fact_book_0708.pdf. [Accessed 18 March 2008]. 42. Ibid., 37. 43. Indiana University Bloomington Libraries, “About the IUB Libraries.” Available online at www.libraries.iub.edu/index.php?pageId=5425. [Accessed 23 August 2007]. 44. Ibid. 45. Paul B. Kantor, Objective Performance Measures for Academic and Research Libraries (Wash- ington, D.C.: Association of Research Libraries, 1984). 46. Blaine H. Hall, Collection Assessment Manual for College and University Libraries (Phoenix, Ariz.: Oryx Press, 1989), 44. A Simulated Electronic Availability Study 443 47. Mansbridge, “Availability Studies in Libraries,” 305. 48. Magazines for Libraries (15th ed.), ed. Cheryl LaGuardia (New Providence, N.J.: CSA, 2007). 49. This potential bias of using citations from dissertations is noted by Penny M. Beile, David N. Boote, and Elizabeth K. Killingsworth, “A Microscope or a Mirror?: A Question of Study Valid- ity Regarding the Use of Dissertation Citation Analysis for Evaluating Research Collections,” but the point would also apply to faculty publications and course reading lists. 50. Everett M. Rogers, Diffusion of Innovations, 5th ed. (New York, Free Press, 2003). 51. Indiana University Bloomington Libraries, “Request Materials from the Library Auxiliary Facility (ALF).” Available online at www.libraries.iub.edu/index.php?pageId=54. [Accessed 9 May 2008]. 52. Mansbridge, “Availability Studies in Libraries,” 305. 53. Nisonger, “A Review and Analysis of Library Availability Studies,” 33. 54. Mansbridge, “Availability Studies in Libraries,” 307. 55. Nisonger, “A Review and Analysis of Library Availability Studies,” 49. 56. Steynberg and Rossouw, “The Availability of Research Journals in South African Academic Medical Libraries,” 838. 57. Shaw-Kokot and Varre, “Using a Journal Availability Study to Improve Access,” 24. 58. Chaudhry and Ashoor, “Comprehensive Materials Availability Studies in Academic Libraries,” 302. 59. Bachmann-Derthick and Spurlock, “Journal Availability at the University of New Mexico,” 197. 60. Roberts, “Journal Availability Study,” 36. 61. Harris and Garner, “Using an Availability Survey to Improve Service at a University Library,” 28. 62. Ciliberti et al., “Empty Handed? A Material Availability Study and Transaction Log Analysis Verification,” 284. 63. Schaffer, “Psychology Citations Revisited,” 354. 64. Price, “How Many Journals Do We Have?” 136. 65. Squires, Moore, and Keesee, “Electronic Journal Availability Study,” 2. 66. These data were sent to the author in an e-mail from Stephen E. Wiberley, Jr. on November 16, 2007. 444 College & Research Libraries September 2009 Appendix A Source Journals for the 50 Subject Areas Area Studies African American Studies—Journal of African American History African Studies—African Studies Review American Studies—American Quarterly Central Eurasian Studies—Central Asiatic Journal East Asian Studies—China Quarterly Jewish Studies—Jewish Studies Quarterly Latin American & Caribbean Studies—Journal of Latin American Studies Russian & East European Studies—Slavic Review West European Studies—European History Quarterly Women’s Studies—Signs: Journal of Women in Culture and Society Humanities Art—Art Journal Classical Studies—Greece and Rome Dance—Dance Research English and American Literature—American Literature Folklore—Journal of Folklore Research French—French Studies Music—Journal of the American Musicological Society Philosophy—American Philosophical Quarterly Religion—Journal of Religion Theatre and Drama—Theatre Journal Professional Fields Business—Academy of Management Review Education—American Educational Research Journal Journalism—Harvard International Journal of Press–Politics Law—Harvard Law Review Library Science—College & Research Libraries Medicine—New England Journal of Medicine Nursing—Nursing Research Optometry—Optometry and Vision Science Public Administration—Public Administration Quarterly Social Work—Social Work Science Astronomy—Astrophysical Journal Biology—FASEB Journal: The Journal of the Federation of American Societies for Experi- mental Biology Chemistry—Journal of the American Chemical Society Computer Science—Journal of the Association for Computer Machinery Environmental Sciences—Environmental Science and Technology Geology—Journal of Metamorphic Geology Information Science—Journal of the American Society for Information Science and Technology Mathematics—American Journal of Mathematics Neuroscience—Nature Reviews Neuroscience Physics—Physical Review A A Simulated Electronic Availability Study 445 Social Sciences Anthropology—American Anthropologist Criminology—Criminology: an Interdisciplinary Journal Economics—Quarterly Journal of Economics Geography—Annals of the Association of American Geographers History—American Historical Review International Relations—International Organization Linguistics—Journal of Linguistics Political Science—American Political Science Review Psychology—Psychological Bulletin Sociology—American Sociological Review