College and Research Libraries The Getty End-User Online ·searching Project in the Humanities: Report No.6: Overview and Conclusions Marcia J. Bates Over a two-year period, the Getty Information Institute (formerly the Getty Art History Information Program) sponsored and carried out a major study of end-user online searching by humanities scholars. Com- plete logs of the searches and output were captured, and the twenty- seven scholars involved were interviewed in depth. An overview of the study and its results is presented, with particular emphasis on matters of interest to academic librarians. Implications are drawn for academic library reference service and collection development, as well as for cata- loging in the online and digital environment. ver a two-year period, the Getty Information Institute (formerly the Getty Art History Information Program, or Getty AHIP) sponsored and carried out a ma- jor study of end-user online searching by humanities scholars. Complete logs of their searches and output were captured, and the twenty-seven scholars involved were interviewed in depth. The results of the study have appeared in a series of five articles to date. 1- 5 It is the purpose of this article to (1) provide an overview of the entire project, (2) re- view major findings of the study, and (3) draw implications of the study research for the practice of academic librarianship in reference and online searching ser- vices, collection development, and cata- loging. Marilyn Schmitt, Deborah Wilde, and Susan Siegfried, of the Getty Information Institute, who originated and carried out much of the study, brought me into the project to design the second year's inter- view schedule and to analyze this im- mense body of data. (Wilde and Siegfried are no longer with the Institute. Vanessa Birdsey, Nancy Bryan, Brian Sullivan, Jeanette Clough, and Katherine Smith also assisted in various parts of the study.) In the space of this necessarily brief article, it is feasible neither to recapitu- late all the results nor, especially, to present all the supporting data that ap- peared in five lengthy articles elsewhere. The purpose here is to extract the infor- mation of use to the busy academic li- brarian; the reader interested in more detail will be directed to the appropriate article(s). As can be seen in the notes, the articles are numbered Reports #1 through 5 to make it easier to distinguish and lo- Marcia J. Bates is a Professor in the Graduate School of Education and Information Studies at the Univer- sity of California, Los Angeles; e-mail: mjbates@ucla.edu . 514 Getty. End-User Online Searching Project 515 cate them; they will also be referred to by number here. Succeeding sections describe the study methodology, state the content areas of each of the five prior reports, extract the principal results from the five prior ar- ticles, and draw implications from these results for academic library service to hu- manities researchers. Background Literature reviews are provided in the various earlier reports, especially #1 through 3. The reader interested in learn- ing more in general about humanities in- formation-seeking is directed to the re- cent review article by Rebecca Watson-Boone,6 the earlier extensive re- view by Sue Stone/ and research by Stephen Wiberley and William Jones.s-9 Methodology Visiting scholars at the Getty Research In- stitute for the History of Art and the Hu- manities10 are invited for a one-year stay, and come from all over the world. To do research at the institute is a prestigious opportunity, and the institute draws a range of qualified researchers, from people finishing their doctorates to in- ternationally renowned senior scholars. In return for the opportunity to do un- limited, free (i.e., subsidized by the Getty) online searching as end users on about 60 humanities and social sciences data- bases, participating scholars agreed to be interviewed and to have their search logs captured for study. During 1988- 89 and 1989-90, all visiting scholars at the re- search institute were invited to partici- pate. Over the two years, about two-thirds of them agreed to participate, as well as five spouses (scholars in their own right), for a total of twenty-seven participants,. seventeen male and ten female . (One of the first-year scholars stayed a second year; see Report #2 for information on where her second-year data are or are not included in various tallies .) Scholars came from all over Europe and the United States. Eleven were native English speakers; English language proficiency of the non-native speakers ranged from ad- equate to excellent. Research interests of the scholars ranged across the arts and humanities, with some working in the social sciences. All took a one-day training session in DIALOG searching with a DIALOG trainer. The scholars also had available some other opportunities for search as- sistance, which they utilized, but, over- all, the study design encouraged schol- ars to do their own searching and fur- ther self-education once the training was over. (See Report #2 for more details on general study design.) Scholars recorded their own natural language descriptions of their queries at the beginning of their searches. A specially written computer program captured these statements, as well as the entire search, including printout results. In other words, full searches and search results were printed out locally, rather than at DIALOG; the program kept a record of (1) the user's natural language query statement (meaningless to DIALOG), (2) Cataloging is now in flux as libraries enter the era of digital resources. all DIALOG search statements by the end user, and (3) all printout retrieval results. The terminology used in natural lan- guage statements and search formula- tions was analyzed in detail. (Method- ological issues associated with terminol- ogy analysis are discussed in Report #1.) An elaborate set of about one hundred codes was developed to analyze the vari- ous aspects of DIALOG searching that was done by the scholars, including use of commands and use of Boolean logic. (The codes are presented in Report #2.) Scholars were also interviewed in depth. (See Report #3 for interview schedules 516 College & Research Libraries used and methodological issues involved in the interviewing.) Overview of Reports to Date The terminology used in the natural lan- guage statements and in search formu- lations was analyzed in detail; these data form the bulk of what is reported on in Report #1. The character of the vocabu- lary, and the implications for cataloging and online searching, were surprising in some ways. Cataloging is now in flux as libraries enter the era of digital resources. Catalogers may be called upon to make substantial changes in descriptive and subject cataloging for this new age. The results in Report #1 may be of particular interest to catalogers designing for new approaches to subject cataloging. An analysis of length and character of the online searches, including use of search commands and Boolean logic, forms the heart of Report #2. Search records also were sampled over time to determine whether the scholars demonstrated a learning curve with experience. Those li- brarians engaged in online searching and bibliographic instruction may find the results in Report #2 revealing. The results of the interviews with scholars are presented in Report #3. Scholars were interviewed on their ex- periences with online searching, the role the searching took in their broader research activities, their reactions to DIA- LOG, and their attitudes about the future of end-user online searching. Reference librarians may find Report #3 of particu- lar interest. As one who has taught science and en- gineering literature and has some sense of the culture and values of the sciences, this author was struck by the distinctive characteristics of the humanities' culture, values, and expectations, in contrast to those of the sciences, as they appeared in our study. These differences are commented on in Reports #1, 4, and 5, in particular. Much of the database world has been de- November 1996 veloped on the ~cience model. Report #4 draws implications from the study for the design of databases and other informa- tion resources for humanities scholars. Librarians may find the points in this re- port of value for their future collection development in reference departments or main collections. Report #5 addresses the possible im- pact of features of online searching in the humanities on the (characteristically sci- ence-based) theory of information sci- ence, especially information retrieval theory and Bradford's Law. Principal Results In this section, the study results of inter- est to academic librarians will be ex- tracted from the various reports and dis- cussed. The points made are not necessarily associated with any one prior report. • The distribution of amount of end- user online searching by the scholars falls out into a familiar pattern of a few using it a lot, and most using it little. It has commonly been found that where end users have the opportunity to learn online searching and search on their own, some few will take it up with en- thusiasm, whereas, in a sharply descend- ing curve of use, many will use it little or not at all after training. 11 This pattern emerged with the Getty scholars. All who participated were required to attend a . one-day training session for DIALOG. Yet, of the twenty-seven scholars, only five spent a total of seven or more hours (each) actually doing online searching over their year's stay after their training session (i.e., only five searched about as long as, or longer than, their training ses- sion lasted) . Five others did no further searching on their own at all after their training session, and most of the rest searched less than two hours. With regard to their background use of computers, the scholars had had van- ishingly little prior experience with on- Getty End-User Online Searching Project 517 line searching, but many had used on- line catalogs before, and most had begun using personal computers for writing: • Positive reactions to online searching dealt mostly with the power of DIALOG to cover many topics or bodies of literature; negative comments dealt mostly with diffi- culty with the command language and the lack of desired resources available online. Scholars liked Dialindex, DIALOG's database of terms covering all databases at once, and OneSearch, which allowed them to search several databases at once. One said, "I feel like an explorer. . . . I enjoy it." Another called online search- ing "Las Vegas for the intellectual," re- ferring to the surprises he found in the output. This same scholar became expe- rienced and comfortable enough with searching that he recognized and com- mented on the power of Boolean logic to search on combinations of two or more topics. Negative comments expressed the beginner's hesitancy in using the system, as well as some of the problems that have been touched on repeatedly by critics of the conventional online Boolean-type systems. The scholars had difficulty con- verting their queries into Boolean logic; they found the DIALOG commands con- fusing, not natural. Searchers found it "too complicated," "daunting," and "cumbersome," and one compared it unfavorably to UCLA's onli!}e catalog, ORION, which he found easier to under- stand. Another scholar said, "I resist what strikes me as a kind of industrialization of scholarship." A separate interview schedule was used for those scholars who had not searched at all during the months after their training session. They said they had little need to do it, and/ or did not want to spend the time experimenting with it. Not finding an hour or two for cost-free ex- perimentation with DIALOG over the course of the better part of a year suggests something more than just lack of time or immediate need-perhaps discomfort or a lack of fundamental interest. Overall, judging by both the scholars' comments and the character of their searching, most did not become skilled enough in online searching to get maxi- mum benefit from it. All search state- ments were coded for "probable errors" and "certain errors." In the case of the latter, something about the search state- ment would have guaranteed failure; with regard to the former, there might conceivably have been some reason to do what the searcher did, but it did not ap- pear to be a wise way to search. Over the two years, 17 percent of all search state- ments contained one or more probable or certain errors, roughly evenly divided be- tween the two classes of error. This is not too bad a rate, considering that most schol- ars did not search enough to build up ex- pertise. Further, errors did not seem to be a major inhibiting factor for the searchers; in fact, one of the most active searchers had the highest personal error rate. Over- all, 32.8 percent of that person's search statements contained one or more errors. At the same time, the searching done was not very sophisticated. Search terms used were mostly (62.5%) single-word terms, and only 3.1 percent of the "Se- lect" statements contained parentheses, which are a mark of more sophisticated searching. However, use of the "build- ing block" technique was emphasized in the scholars' training. This technique of entering the elements, or building blocks, of the query one at a time is an excellent one for beginners to use, and may ac- count for the apparent simplicity of the scholars' search statements. Three of the twenty-two scholars who searched subsequent to their training never used a single Boolean operator; another five used only ANDs when they did do Boolean searching. So these eight, a third of the active searchers, were un- able to take advantage of the common searcher technique of inputting "hedges" (strings of terms ORed together) of mul- 518 College & Research Libraries tiple term variants for a single concept in order to get full coverage of a topic. The other main body of scholars' criti- cisms of online searching concerned the contents of the databases. Available data- bases often proved not to be what the scholars wanted. They wanted access to more European literature, earlier literature, and primary research materials. Most da- tabases cover only literature going back to the early 1970s, which is often just a small part of the range that a humanities scholar is interested in. Earlier literature is generally not in machine-readable form, so is much more expensive for the vendor to mount. (See Report #4 for some sug- gested ways to meet both scholar needs and vendor interest in the bottom line.) • For these scholars, use of DIALOG searching was valuable largely at the mar- gins. There were a number of spontaneous comments to the effect that the scholar searchers were pleased to find something of value at all in the searching-not be- The other main body of scholars'. criticisms of online searching concerned the contents of the databases. cause they doubted the effectiveness of the retrieval system but, rather, because they considered themselves experts on their subjects of interest and therefore did not expect to come across anything they did not already know. The standard assumption behind much of information retrieval theory is that the person making a query will be familiar with little or none of the litera- ture found in a bibliographic search. In fact, this assumption is so fundamental that it is hardly ever articulated. It is just assumed that people would not be searching if they already had found what they wanted. That may be reasonably ac- November 1996 curate for many nonhumanities research- ers, but for a variety of reasons having to do with fundamental differences in the very nature of humanities scholarship and scientific research (discussed in depth in Reports #4 and #5), it may be more common for the scholar to know, and know very thoroughly, the literature respecting his or her subject of inquiry, than is the case for the scientist. Other research on humanities schol- ars has found that they use the indexes that are the basis for bibliographic data- bases little in print form either, and gather much of their key references through a variety of other channels, such as refer- ences in articles, specialized bibliogra- phies, colleagues, etc. 12- 13 One might nonetheless make the straightforward assumption that once scholars had dis- covered the power and scope of online database searching, they would happily move to it and abandon older, more scattershot methods. This may yet hap- pen, but did not occur during the year the scholars had their free opportunity, with the possible exception of one or two of the most enthusiastic searchers. Another reason for low usage could have been that the scholars already had done their research before arriving, and came only to write during their year at the Getty. However, this proved not to be the case. The fifteen scholar-partici- pants during the second year were asked about the ~tatus of their research projects. It turned out that the fifteen were en- gaged in thirty-nine different projects among them, at all stages of develop- ment. No discernible association was found between stage of project and amount of online searching. Upon being asked, several of the schol- ars said they expected that online data- bases would raise the expectations of thor- oughness in scholarly preparation in the field. Nonetheless, at the time of their in- terview, only two scholars had actually looked into how they might be able to con- tinue searching once they returned home. Getty End-User Online Searching Project 519 Five scholars mentioned the value of online searching to explore interdiscipli- nary topics or topics in neighboring dis- ciplines. So it would appear that in a very clear-cut way, the use of DIALOG served indeed to benefit the scholars at the mar- gins of their interests. They often found little at the core of their interest area(s), because they had developed extensive bib- liographies of materials in those areas over the years, and over many projects . and papers. But the power of online searching enabled them to discover their topic be- ing discussed in another field, and some of the best finds were materials found outside their own discipline. • Scholars' search mode preferences var- ied over five distinct options. One of the most striking results of the study was the wide range of preferences the scholars expressed for modes of on- line searching. The following question was asked of the second year's fifteen visiting scholars: "Do you prefer to do online database searching yourself, or would you rather have librarians or re- search assistants do it for you?" The interview style was conversa- tional, and scholars responded in a vari- ety of ways to this question. Their pref- erences can be summarized as follows, with searcher referring to those who searched by themselves at some point after their training session and non- searcher referring to those who did not search by themselves after their training session: • self-searching alone: two searchers, one nonsearcher; • self with somebody nearby to answer questions: four searchers, zero nonsearchers; • self with someone sitting next to one: two searchers, one nonsearcher; • intermediary searching with scholar sit- ting there giving feedback: two searchers, one nonsearcher; • intermediary searching without scholar present: one searcher, one nonsearcher; . • no response: one searcher. This remarkable range of preferences suggests that librarians doing online searching may need to expect to work out a variety of arrangements with their us- ers, rather than assuming that one ar- rangement will work best. • The logical, engineering-oriented de- sign of online systems is generally not well matched with the talents of the humanities scholar. The humanities scholar is a genius at detecting the trend or nuance that is not explicitly expressed in the text being studied, at reading through and between the lines. The design of current online re- trieval systems, by contrast, is intended to be explicit, rigorously consistent, and logico-mathematical in design. In analyz- ing the data, the researchers were struck, again and again, by the "simple" logical and other errors made by these brilliant scholars as they attempted to master DIALOG searching. One small example is telling: In the DIALOG manual available at the time (this has since been changed), the "N" operator was shown as "nN." The "N" operator is one with which the searcher can ask the system to search for two words in either direct or reverse order. The "nN" model was meant to show that the searcher could put a number before the N operator to indicate how many words could be between the two search words and still retrieve the record. For example, "Child (lN) psychology" would mean that the phrases "Child psy- chology" and "Psychology of children," among others, would both be retrieved by such a formulation. One of the schol- ars input this operator directly as "nN" in a search, however, without convert- ing the "n" to a number, and conse- quently retrieved nothing. Now this usage is essentially alge- braic. The small letter can be substituted for by many numbers. In the author's ex- perience teaching online searching, she has been struck by the ease with which students with a strong mathematics or 520 College & Research Libraries engineering background pick up Boolean searching, and the difficulty that many, though not all, people with a humani- ties background have in learning this skill. The humanities scholar might well point out that the engineer is never again required to repeat the poem he or she memorized in ninth-grade English. Why, then, should the humanities scholar be required to remember ninth-grade alge- bra? The humanities scholar will experience online searching differently. Referring to online searching as "the industrialization of scholarship," as mentioned earlier, is the sort of conceptualization of the searching experience not likely to be foremost in the mind of the engineer. Another scholar liked the idea of having her own pass- word, which, she said, "is a kind of inti- mate relation to the machine." Others were conscious of a sense of being connected in to a network-one scholar comment- ing positively on that sensation, and one negatively. This sophisticated awareness of social and aesthetic dimensions of this particular human activity of online search- ing is bound to have some impact on the reactions of scholars to its use. • The character of humanities search terms varies considerably from that of the sci- ences. Humanities thesauri probably should be designed on different principles from con- ventional thesauri, and humanities search in- terfaces should be designed differently as well. Stephen Wiberley argued, beginning in 1983, that humanities subject terms are mostly not the characteristically vague, hard-to-define terms that they are gen- erally assumed to be. 14-15 He provided convincing evidence that terms of impor- tance to humanities scholars in reference books and indexes are often highly pre- cise proper names. In this study, both the vocabulary in the natural language descriptions of que- ries given by the scholars, as well as the actual terms used in searching, were ana- lyzed in detail (Report #1). The data con- November 1996 firmed Wiberley's findings and demon- strated a number of other differences from conventional thesaural vocabulary and search vocabulary as well. In the natural language statements, the scholars' descriptions of their queries were expressed in the most natural, na- tive way, not yet converted into search statements. Ninety-one percent, or 150, of the 165 natural language statements indicated a subject search of some kind. These subject queries were the focus of the author's analysis. After reviewing the data, these catego- ries of subject terms were developed: (1) works or publications as subject, (2) indi- viduals as subject, (3) geographical term, (4) chronological (date) term, (5) discipline term, (6) other proper term, and (7) other common term. "Discipline term" refers to terms comparable to academic disci- plines, such as "history," "philosophy." "Other proper" refers to proper nouns (normally capitalized words) that do not fall into any of the preceding categories, and "other common" referred to any common (uncapi-talized) terms that do not fall into the preceding categories. Data analyzed from these 150 subject statements were then compared to the text of thirty-eight real natural and so- cial science queries that had been gath- ered for a National Science Foundation- funded study of online searching carried out by Tefko Saracevic and others. 16 For the researchers' study, the Saracevic que- ries were analyzed in the same way as the Getty natural language statements. The data are summarized in table 1. The figures in each row represent the percent- age of queries that contained one or more terms of the designated type. (More de- tail on the methodology of counting these terms is found in Report #1.) An indefi- nite number of terms and term types can be in a single query. The dramatic difference in termino- logical profile between the two bodies of data is immediately evident in the table. All the Saracevic queries contained one Getty End-User Online Searching Project 521 TABLE 1 or more common terms; only 57 percent of the Getty queries did. Consider a typical Saracevic study query: "the relationship and communication pro- cesses between middle- aged children and their parents." 17 All terms in this query are common; there are none of any other type. This is typi- cal of the science que- ries. Note what small Frequencies of Subject Categories in NSF and Getty Studies NSF Getty N % N % Total subject queries 38 100% 150 100% Works or pubs. as subject 1 3 8 5 Individuals as subject 0 0 74 49 Geographical name 3 8 37 25 Chronological term 1 3 26 17 Discipline term 0 0 35 23 Other proper term 4 11 11 7 Other common term 38 100 85 57 percentages of the que- Note: Percentages are the percentage of total query statements in each ries in the Saracevic sample in which one or more terms of a given category appeared. sample contain any other type of term. Now consider one of the Getty queries: "image of the tree in literature, art, science, of medieval and renaissance Europe." This query contains geographical, chronologi- cal, and discipline terms, in addition to the common terms. As noted above, only 57 percent of the Getty statements contained any common terms at all. On the other hand, high rates of all the noncommon term types appear in the Getty column, in sharp contrast to the data in the Saracevic column. This general pattern carried over into the actual search terms used by the scholars as well. Much of the theory of thesaurus de- sign focuses on the development of what are here called "common" terms. Given the profile of the science data, and the historic importance of science databases in online searching, this is understand- able and reasonable. But it is also clear that if the needs of humanities scholar- ship are to be well served, much more attention needs to be given to the design of terminology of the other types as well. (The study results inadvertently con- firmed the value of two Getty databases that were already in development at the time of the study, The Union List of Artist Names and The Thesaurus of Geographic Names.) Good design of such terminology in the online searching context is not obvious. Con- sider dates, for example, which would seem to be the most straightforward pos- sible type of search term. However, the producers of the database Historical Ab- stracts experimented for several years be- fore finding optimal ways of coding and searching on dates to provide good re- trieval under many circumstances. (See Bates for a more extensive discussion.18) The broad categories of subject term found in this study lend themselves well to a faceted approach to index terminol- ogy. (The choices that had been made ear- lier to design the Getty's Art & Architec- ture Thesaurus on a faceted basis are strongly confirmed by this study's data.) Likewise, the scholars frequently at- tempted to create search terms that were, in effect, strings of facet elements. Un- fortunately, because they did not fully un- derstand the use of Boolean logic and proximity operators, and because current indexing is often not well suited to such strongly faceted subject matter, they fre- quently produced strings that would not work online, for instance: "humanities(w)(method? OR me- thodology)(w)(comparison usa europe)" 522 College & Research Libraries Suppose that the searcher could in- stead have entered elements of this query into a prompt screen with labeled slots for geographical term, date, topic, named individuals, etc. In that case, the system could carry out the Boolean work; the user need only know what he or she knows best-the elements of the query. The indexing in many current data- bases does not recognize and build on these facet elements well, thus making it difficult to form a good search statement, even when one is an expert searcher. Many of the humanities queries work like the example above-that is, they consist of several elements, each one of which is conceptually quite broad ("humanities," "Europe"), but when the elements are all brought together (in effect, ANDed), they form a highly specific query. Yet, pre- cisely because of the breadth of many of these terms, indexers are discouraged from using them. Database producers need to recognize that with queries of the distinctive sort found in the humanities, indexing terms and policies may need some adaptation to produce indexing that is optimally effective for online users. Implications for Library Service Keeping in mind the relatively small sample upon which these conclusions are drawn, consider the following points re- garding academic library services for humanities scholars: November 1996 • Because online searching has been powerful in discovering work of interest for scholars in other disciplines, give par- ticular attention to providing interdisci- plinary information in online searching for humanities scholars. Use multidis- ciplinary databases such as DIALOG's Dialindex and lSI's citation databases. • Recognize that humanities searches are often composed of several facet ele- ments. Because the indexing for many databases is not optimally designed for queries of this sort, good online database searching in the humanities may actually be harder than for the sciences, even for the skilled online intermediary, and will almost certainly be difficult for the typi- cal humanities end user. • Expect a wide array of preferences among humanities users for database searching arrangements-from the scholar searching alone, through various mixes of scholar and intermediary working to- gether, to the intermediary doing the search alone for the scholar. • Look to acquire reference materials in CD-ROM or other automated forms that show a sensitivity to the unique charac- teristics of humanities scholars and schol- arship, specifically, that provide indexing sensitive to the typical kinds of facets of interest to scholars, that have very user- friendly interfaces, and that contain re- sources with historical and geographical depth. Notes 1. Marcia J. Bates, Deborah N. Wilde, and Susan Siegfried, "An Analysis of Search Terminol- ogy Used by Humanities Scholars: The Getty Online Searching Project Report Number 1," Library Quarterly 63 (Jan. 1993): 1-39. 2. Susan Siegfried, Marcia J. Bates, and Deborah N. Wilde, "A Profile of End- User Searching Behavior by Humanities Scholars: The Getty Online Searching Project Report No.2," Journal of the American Society for Information Science 44 (June 1993): 273-91. 3. Marcia J. Bates, Deborah N. Wilde, and Susan Siegfried, "Research Practices of Humanities Scholars in an Online Environment: The Getty Online Searching Project Report No.3," Library and Information Science Research 17 (winter 1995): 5-40. 4. Marcia J. Bates, "The Design of Databases and Other Information Resources for Humani- ties Scholars: The Getty Online Searching Project Report No. 4," Online and CD-ROM Review 18 (Dec. 1994): 331-40. 5. --, "Document Familiarity, Relevance, and Bradford's Law: The Getty Online Search- ing Project Report No. 5," Information Processing & Management 32 (November 1996): in press. Getty End-User Online Searching Project 523 6. Rebecca Watson-Boone, "The Information Needs and Habits of Humanities Scholars," RQ 34 (winter 1994): 203-16. 7. Sue Stone, "Humanities Scholars: Information Needs and Uses," Journal of Documentation 38 (Dec. 1982): 292-312. 8. Stephen E. Wiberley Jr. and William G. Jones, "Patterns of Information Seeking in the Hu- manities," College & Research Libraries 50 (Nov. 1989): 638-45. 9. ---, "Humanists Revisited: A Longitudinal Look at the Adoption of Information Tech- nology,'' College & Research Libraries 55 (Nov. 1994): 499-509. 10. The J. Paul Getty Trust is a private operating foundation . Two of its programs are the Getty Information Institute (formerly the Getty Art History Information Program) and the Getty Re- search Institute for the History of Art and the Humanities (formerly the Getty Center for the His- tory of Art and the Humanities). These two entities collaborated on this research. 11. Bryan Pfaffenberger, Democratizing Information: Online Databases and the Rise of End-User Searching (Boston: G. K. Hall, 1990), 111-12. 12. Wiberley and Jones, "Patterns of Information Seeking." 13. Susan S. Guest, "The Use of Bibliographical Tools by Humanities Faculty at the State Uni- versity of New York at Albany," Reference Librarian 18 (summer 1987): 157-72. 14. Stephen E. Wiberley Jr., "Subject Access in the Humanities and the Precision of the Humanist's Vocabulary," Library Quarterly 53 (Oct. 1983): 420-33. 15. ---, "Names in Space and Time: The Indexing Vocabulary of the Humanities," Library Quarterly 58 (Jan. 1988): 1-28. 16. Tefko Saracevic and Paul Kantor, "A Study of Information Seeking and Retrieving. II. Us- ers, Questions, and Effectiveness," Journal of the American Society for Information Science 39 (May 1988): 177-96. 17. Ibid., 195. 18. Marcia J. Bates, "Implications of the Subject Subdivisions Conference: The Shift in Online Catalog Design," in The Future of Subdivisions in the Library of Congress Subject Headings System: Report from the Subject Subdivisions Conference Sponsored by the Library of Congress, May 9-12, 1991, ed. Martha 0. Conway (Washington, D.C.: Library of Congress Cataloging Distribution Service, 1992), 92-98. Statement of ownership, management, and circulation College & Research Libraries, ISSN 0010-0870, is published bimonthly by the Association of Col- lege and Research Libraries, American Library Association, 50 E. Huron St., Chicago, IL 60611- 2795. The editor is Donald Riggs, University of Michigan Library, 818 Hatcher Graduate Li- brary, Ann Arbor, MI 48109-1185. Annual subscription price, 55.00. Printed in U.S.A. with second-class postage paid at Chicago, Illinois. As a nonprofit organization authorized to mail at special rates (DMM Section 424.12 on1y), the purpose, function, and nonprofit status of this organization and the exempt status for federal income tax purposes have not changed during the preceding twelve months. Extent and nature of circulation (Average figures denote the average number of copies printed each issue during the preceding twelve months; actual figures denote actual number of copies of single issue published nearest filing date: September 1996 issue.) Total number of copies printed: average 12,703; actual12,613. Sales through dealers, carriers, street vendors, and counter sales: none. Mail subscription: average 12,276; actual12,262. Free distribution: average 284; actual296. Total distribution: average 12,560; actual12,558. Office use, leftover, unaccounted, spoiled after printing: average 143; actual 55. Total: average 12,703; actual 12,613. Statement of Ownership, Management, and Circulation (PS Form 3526, Oct. 1994) for 1996 filed with the United States Post Office Postmaster in Chicago, October 1, 1996. REFERENcE -~GREENWOOD PRESS Literary Journalism A Biographical Dictionary of Writers and Editors By Edd Applegate Overviews literary journalism and provides bio- graphical entries for writers and editors who prac- ticed literary journalism. Greenwood Press. 1996. 352 pages. 0-313 -29949-8. $85 .00. An Encyclopedia of the History of Classical Archaeology Edited by Nancy Thomson de Grummond The first encyclopedia on the history of classical archaeology, this volume includes 1, 125 entries on Greek and Roman material, the Bronze Age Aegean, the Etruscans, and Greek and Roman remains in France and Asia Minor. Greenwood Press. 1996. 1400 pages. 2 vols. 0-313-22066-2. $225 .00. Makers of 20th-Century Modern Architecture A Bio-Critical Sourcebook By Donald Leslie Johnson and Donald Langmead The most influential 20th century architects espous- ing modernism are brought together in critical dis- cussion and independent profiles . Greenwood Press. 1996. 440 pages. 0-313 -29353-8 . $89.50. Handbook of Twentieth- Century Literatures of India Edited by Nalini Natarajan Surveys the many regional literatures of 20th centu- ry India. Greenwood Press. 1996. 448 pages. 0-313-28778-3 . $85 .00. The American Civil War A Handbook of Literature and Research Edited by Steven E. Woodworth Foreword by James M. McPherson "The single most important volume for anyone interested in the Civil War to own and consult." From the foreword by James M. McPherson Dictionary of Irish Literature Revised and Expanded Edition Robert Hogan, Editor-in-Chief An ALA Outstanding Reference Book, 1978 (first edition) A Choice Outstanding Academic Book, 1980 (first edition) Associate Editors: Zack Bowen, Richard Burnham, Mary Rose Callaghan, Anne Colman, Peter Costello, William J. Feeney, Maryanne Felter, James Kilroy, Bernard McKenna, and Marguerite Quintelli-Neary "A uniformly well written reference on the major and minor figures and institutions of modem Anglo-Irish literature ... Recommended for both public and academic libraries." Library Journal on the first edition Greenwood Press. 1996. 14 72 pages. 2 vols. 0-313-29172-1. $135 .00 . Every Thing in Dickens Ideas and Subjects Discussed by Charles Dickens in His Complete Works A Topicon Compiled and Edited by George Newlin The greatest quotes from Dickens. . . an essential reference book providing every notable and quotable passage or short comment by Dickens on a subject which interested the great author . .. encom- passing all his work. Greenwood Press. 1996. 1168 pages. 0-313-29874-2 . $145 .00 . Encyclopedia of Classical Philosophy Edited by Donald J. Zeyl Associate Editors: Daniel T. Devereux and Phillip K. Mitsis The only encyclopedia in English specific to the field of Classical Philosophy. Over 250 articles on major and minor figures and topics by leading scholars . Greenwood Press. 1996. 0-313-28775-9 . $99.00 Est. 1 PLACE YOUR CREDIT CARD ORDER TOLL-FREE 24 HOURS-A-DAY: 1-800-225-5800