Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. The Impact of Web Search Engines on Subject Searching in OPAC Yu, Holly;Young, Margo Information Technology and Libraries; Dec 2004; 23, 4; ProQuest pg. 168 The Impact of Web Search Engines on Subject Searching in OPAC Holly Yu and Margo Young This paper analyzes the results of transaction logs at California State University, Los Angeles (CSULA) and studies the effects of implementing a Web-based OPAC along with interface changes. The authors find that user success in subject searching remains problematic. A major increase in the frequency of searches that would have been more successful in resources other than the library catalog is noted over the time period 2000-2002. The authors attribute this increase to the prevalence of Web search engines and suggest that metasearching, relevance-ranked results, and relevance feedback ("more like this") are now expected in user searching and should be integrated into online catalogs as search options. I n spite of many studies and articles on Online Public Access Catalogs (OPAC) over the last twenty-five years, many of the original ideas about improving user success in searching library catalog have yet to be imple- mented. Ironically, many of these techniques are now found in Web search engines. The popularity of the Web appears to have influenced users' mental models and thus their expectations and behavior when using a Web- based OPAC interface. This study examines current search behavior using transaction-log analysis (TLA) of subject searches when zero-hits are retrieved. It considers some of the features of Web search engines and online bookstores and suggests future enhancements for OPACs. I Literature Review Many studies have been published since the 1980s center- ing on the OPAC. Seymour and Large and Beheshti pro- vide in-depth overviews on OPAC research from the mid-1980s through the mid-1990s.' Much of this research has addressed system design and user behavior including: • user demographic s, • search behavior, • knowledge of system, • knowledge of subject matter, Holly Yu (hyu3@calstatela.edu) is Library Web Administrator and Reference Librarian at the University Library, California State University, Los Angeles. Margo Young (Margo.e.young@jpl. nasa.gov) is Manager of the Library, Archives and Records Sec- tion at the Jet Propulsion Laboratory, California Institute of Technology, Pasadena. • library settings, • search strategies, and • OPAC systems 2 OPAC research has employed a number of data-col- lection methodologies: experiment, interviews, question- naires, observation, think aloud, and transaction logs. ' Transaction logs have been used extensively to study the use of OPACs, and library literature reflects this. While the exact details of TLA vary greatly, Peters et al. define it simply as "the study of electronically recorded interac- tions between online information retrieval systems and the persons who search for the information found in those systems."' This section reviews the TLA literature relevant to the study. I Number of Hits TLA cannot portray user intention or actual satisfaction since relevance, success, or failure are subjectively deter- mined and require the user to decide. Peters recommends combining TLA with another technique such as observa- tion, questionnaire or survey, interview, or focus group. 5 In spite of the limit ations of TLA, many studies (including this one) rely on it alone. Typically, these studies define failure as zero hits in response to a search. Generalizing from several studies, approximately 30 percent of all searches result in zero hits.6 The failure rate is even higher for subject searches: Peters reported that about 40 percent of subject searches failed by retrieving zero hits. 7 Some researchers also define an upper number of results for a successful sea rch. Buckland found that the average retrieval set was 98.8 Blecic reported that Cochrane and Markey found that OPAC users retrieve too much (15 percent of the time). 9 Wiberly, Daugherty, and Danowski (as reported in Peters) found that the median number of postings considered to be too many was fifteen, although when fifteen to thirty postings were retrieved, more users displayed them all than abandoned the search. 10 I Subject Searching Some studies have specifically looked at subject search- ing. Hildreth differentiated among various types of searches and defined one hundred items as the upper limit for keyword searches and ninety as the upper limit for subject searches." Larson defined reasonable subject retrieval as between one and twenty items and found that only 12 percent of subject searches retrieved the appro- priate number. 12 168 INFORMATION TECHNOLOGY AND LIBRARIES I DECEMBER 2004 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Larson is not the only researcher to have reported poor results in subject searching. For more than twenty years, research has demonstrated that subject or topical searches are both popular and problematic. Tolle and Han found that subject searching is most frequently used and the least successful. 13 Moore reported that 30 percent of searches were for subject, and Matthews et al. found that 59 percent of all searches were for subject information. 14 Hunter found that 52 percent of all searches were subject searches and that 63 percent of these had zero hits. 15 Van Pulis and Ludy referred to Alzofon and Van Pulis's earlier work in 1984 where they reported that 42 percent of all searches were subject searches.16 Hildreth found that 62.1 percent of subject searches and 35.4 percent of keyword searches failed. 17 Larson categorized the major problems with online catalogs as follows: • users' lack of knowledge of Library of Congress sub- ject headings (LCSH), • users' problems with mechanical and conceptual aspects of query formulation, • searches that retrieve nothing, • searches that retrieve too much, and • searches that retrieve records that do not match what the user had in mind. 18 During an eleven-year longitudinal study, Larson found that subject searching was being replaced by key- word searching. 19 No consistent pattern in the number of search terms has emerged in the literature. Van Pulis and Ludy reported that user searches were typically single words. 20 Markey contended that users' search terms frequently matched standardized vocabulary in large catalogs. 21 None of Markey's researchers consulted LCSH, and only 11 percent of Van Pulis and Ludy's did so, notably in spite of their library's user-education programs. Peters reported that Lester found that the average search was less than two words and fewer than thirteen characters." Hildreth found that more than two-thirds of keyword searches included two or more words and 42 percent of these multiple-word searches resulted in zero hits. 23 The proportion of zero-hit keyword searches rose with the increasing number of words in the search. Subject headings have been a matter of considerable study. Gerhan examined catalog records and surmised their accessibility in an online catalog. He contended that when a keyword from the title only is accessed, only 50 percent of all relevant books would be found and that title keywords would lead a user to subject-relevant records in 55 percent of cases while LCSH would lead a user success- fully in 85 percent of the cases. 24 In contrast, Cherry found that 42 percent of zero-hit subject searches would have been more fruitful as keyword or title searches than by fol- lowing cross references retrieved from the subject field.25 She recommended converting zero-hit subject queries to other types of subject searches (keyword). Thorne and Whitlatch recommended that subject searchers should select keyword rather than subject headings as their first access strategy. 26 Types of Problems in Subject Searches Numerous studies have categorized reasons for search failure (typically in zero-hit situations), but Peters reports that a standard categorization has not yet been estab- lished .27 Tn cases where more than one error is made in a search (and Hunter reported this to be frequent), there is no consistency in how that is assigned. Nonetheless, some major categories of problems stand out: • misspelling and typographical errors-Peters found that these errors accounted for 20.8 percent of all unsuccessful keyword searches, while Henty (reported by Peters) concluded that 33 percent of such searches could be attributed to this.28 Hunter found that 9.3 per- cent of subject searches had typographical and spelling errors. 29 • keyword search-Hunter found 52.6 percent of zero- hit searches used uncontrolled vocabulary terms. 30 • wrong source or field-Hunter concluded that 4.5 percent of searches should have been done in a source other than the catalog, while 1.3 percent of searches were of the wrong type (an author search in the subject-search option). 31 • items not in the database-Peters found that searches for items not held in the database accounted for 39.1 percent of unsuccessful searches, while Hunter found that problem in only 2.5 percent of the problem cases. 32 In addition to these problems, Hunter also found that index display and rules relating to the systems accounted for 27 percent of errors. 33 I Resulting Recommendations for Change While Hildreth stated, "There has been little research on most components of the OPAC interface" in 1997, he pro- posed two options to improve user success: increased user training or improved design based on information- seeking behavior. 34 Wallace pointed out that there is a very short window of opportunity when searchers are amenable to instruction and that successful screen designs should therefore focus on presenting the quick- searching options employed by the majority of users first. 35 Large and Beheshti observed "that too many options simply caused confusion, at least for less experi- enced OPAC users," and they summarized that OPAC- THE IMPACT OF WEB SEARCH ENGINES ON SUBJECT SEARCHING IN OPAC I YU AND YOUNG 169 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. interface research focuses on menu sequence, browsing, and querying .3'; Menu Sequence In terms of menu sequence, Hancock-Beaulieu indicated that "the menu sequence in which search options are offered will influence user selection." 37 Ballard found that the amount of keyword searching was affected by its posi - tion on the menu. 38 Scott reported that both keyword- and subject-search success improved when the keyword was plac ed at the top of the menus .39 Thorne and Whitlach used a combination of methods in their study and concluded that several interface changes should be implemented : • strongly encourage novi ce users to start with key- word (list keyword above subject heading), • relabel "keyword" to "subject or title words," and • relabel II subject heading" to "Library of Congress Subject Heading."' 0 Blecic et al. studied tran sactio n logs over six months to track th e impact of "simplifying and clarifying" OPAC introductory screens. After moving the keyword option to th e top, keyword searching incr ease d from 13.30 per- cent to 15.83 percent of all sea rch statements. Blecic et al. found her original tally of 35.05 p ercent of correct searches having zero hits decre ased to 31.35 percent after screen changes. 41 Querying OPAC-interface design has been based on an assumption that us ers come to the catalog knowing what the y need to know . In either text-bas ed OPAC or Web-based OPAC, query-based searches are still mainstream. Searchers are required to have knowledge of title, author, or subject. Ortiz-Repiso and Moscoso observed that Web-based cata- logs, like all library catalogs, basi cally fulfill two functions: locating works based on known details and identifying which documents in the databas e cover a given subject. 42 Natural-language input has long been considered a desi r- able way to overcome this shortcoming. Browsing Relevance-ranked output and hypertext were considered by Hildr eth to be promising in 1997.43 OPACs have not been conceived within a true h ypertext environment, but rather they maintain the structure of their original for- mats, principally machine-readable cataloging (MARC), and therefore impede the generation of a structure of nodes and links. 44 In addition to continuing to employ MARC format as its underlying structure, the concept of main entry and added entr y, field label, and displa y logic all reflect cataloging rules . Amazon.com and Barnes and Noble have completel y mo ve d away from this century- old structure to pro vi d e easy access to book information . In the Web environment , th e concept of main ent ry loses its meaning to multiple-acces s points and linking capabil- ities of author, subject, and call number. Another prominent drawback of Web-based OP A Cs is that they have not taken advantage of thesaurus structure and utilized the thesaurus for sea rching feedback. The hierarc hical relationship in LCSH is underutilized in terms of the relationship betw een terms and associations through related terms. Web-based OPACs have failed to make use of this important access. The persistence of the se drawbacks in OPAC-interfac e design is rooted d eeply in cataloging rules that were derived from the manual environment more than a cen- tury ago. It reflects th e gap between "concepts typically h eld by nonprofessional users and those used in library practices." 45 In her article "Why Are Online Catalogs Still Hard to Use?" Borgman conclude s: Despite numerous imp rove m en ts to the user interfac e of online catalogs in recent years, searc her s still find th em hard to use . Most of th e improvements are in sur- face features rather than in th e core functionality. We see little evidence th at our research on searching behav- ior studies has influ enced onlin e catalog design. " Catalog Content Users misunderstand th e scope of the catalog. In ques- tionnaire responses, 80 percent of Van Pulis and Ludy 's participants indicated the y had considered looking else- where than the library catalog, as in periodical ind exes. 47 Blazek and Bilal report ed a reque st for inclusion of journal- article titles in one respo nse to their questionnaire .48 Libraries responded to th ese requests by acquiring data- bas es on CD-ROM , loadin g them locally (sometimes using the catalog system to mount a separate databas e), and, most recently, providing access to databases over the Internet. However, seldom h ave libraries responded to these requests by integratin g searc h access through a sin- gle front end as the default search. I Impact of Web Search Engines Blecic et al. found that keyword searching increased from 13.3 percent to 28.3 percent over her four-year series of logs. At the same time, zero hits in keyword increased from 8.71 percent to 20.78 percent while subject zero hits dropped from 23 percent to 13.69 percent. She surmised th at the influence of Web interfaces might have affected the regression-fluctuation in search syntax, initial articles, and author order. 49 170 INFORMATION TECHNOLOGY ANO LIBRARIES I DECEMBER 2004 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. automatically sco uts the Web for pa ges that are related to its res ults so it can find a large number of resources ver y qu ickly without requiring th e user to select the right keyw ord s . Teoma structures the appropriate com- munities of int eres t on-the-fly and ranks th e results on a range of facto rs including authorities and hubs (good resources pointing to related resources). Google offers an opti on of "similar pages." Whil e the subj ect-r edirect function in a Web-bas ed OPAC emulates thi s, it succ ee ds only if th e user 's initi a l search term y ielded the right result. OPAC u sers ha ve the option of clicking on hyper- linked h ea din gs (author, titl e, subject headin gs ) but can- not ask the sys tem to perform a more so phisticated sea rch on their behalf. User-Popularity Tracking Amazon and Barnes and Nobl e Web sites pr ese nt enhan- ced information about items b y user-popul arity tracking. Circulation stati stics or user comments could serv e as a form of "r ecommend er sys tem " to h elp novi ces narrow th eir selections. Messa ge s such as "o ther student s who checked this bo ok out also read thes e book s" could be dynamically in serted in bibliographic records. Users could also be allowed to pro vide comment s on mat eri als in the catalog, thus providin g an int era ctive experience for OPAC user s. Summary of Web Features There are positive and negati ve imp acts of Web sea rch engines and on line bookstores on Web-based OPAC u sers . U sers who find Web p ages to b e comfortable, easy, and familiar may mak e greater use of Web-ba sed OPACs. While th ey brin g with them their knowledge of search eng ine s, they also brin g their misp erce ption s. The possi- bility of using similar too ls to those found on Web sea rch en gine s can greatly "re infor ce the u sefuln ess of the cata- log as well as th e positiv e perc eption that th e end us er has of it." 61 Given the diver sity of the error s that u sers experi- ence , a co mbination of approaches is necessa r y to improve their search success. Automatic mapping of free- tex t-to -th esauru s term s, tran sla tion of common spelling mistak es, and links to related pages are to ols alr eady in use in th e Web sea rch engines . "See similar pages," exten- sive us e of releva nce feedback, and popularity track ing along with natural language are less common. I Recommendations for Web-based OPACs Th e authors' TLA rev ea led a continuing problem with subject-h ea din g searches and sho we d a trend toward searching top ics that are n ot typically answered in a bo ok catalog. The form er probl em ha s a well-documented hi s- tory, whil e the authors b elieve th e latt er probl em stems from the influence of th e Web and Web sear ch engin es . Severa l changes to typical OPACs are recommended to addr ess th e trend s observ ed in th e cour se of thi s study. Metasearching Th e recent trend of incorporating databases and OPAC s into a single sear ch reflects the neces sity of exp anding information resourc es and simplif ying access to resources. Thi s stud y's empirical results clearly indicate a need to exp and thi s integration into one sea rch. While some argu e that this metasearching w ill further au gment the syntax digr ess ion an d pr eve nt us ers from becom ing information literate, oth er s beli eve that metas ea rchin g, along with th e option of sear chin g each individu al d a tabase , is an ulti- m a te goal for onl ine search. Like it or not, the m etasearch technolog y, also known as federat ed or broadca st search, "crea tes a portal that could allow the libr ary to become the on e-stop shop th eir us ers and potential use rs find so attractive ."65 One- sea rch-for-all cannot solve all problems; how ev er, guidin g u sers to where the y are mo st likel y to find results quickly (the quick search) should sa tisfy th e ne ed s of th e majority of u sers . Menu Sequence Eff ec tive scree n d es ign h as a p osi ti ve e ffec t on user su c- cess. The m enu sequence for search opti ons plays a signif- icant role in user selection . This research and oth ers it h ave demonstr ated th at users choose an option hi gher rath er than lower in a list. Too many options "simply cause con- fu sion, at least for less experienced OPAC use rs." •• Browsing Feature Browsing is a natural and effective approach to m any information -seekin g problems an d requires less effort and knowledge on the part of th e u ser. The liter a ture sug- gests that a great deal of the use of th e Web relies on known Web sites, recommended sites , or return visits to sit es recently visited-thus relying on browsin g rather than on searching. Jenkins, Co rritore , and Widenb eck found that domain novice s seldom clicked very deep- out and b ack-while Web experts explor ed mor e deeply. 67 Holscher and Strub e not e that Hurtineene and Wandtke claim that only minimal trainin g is necessa ry for brow s- in g an individual Web site, whil e Pollok and Hockl ey claim that considerably more experience is req uired for qu ery ing and na viga tin g among sites. 68 Hancock -Beaulieu found that betwe en 30 p ercent and 45 percent of all online searches, reg ardl ess of th e typ e of search, ar e concluded with brow sing the librar y shelve s.69 176 INFORMATION TECHNOLOGY AND LIBRARIES I DECEMBER 2004 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. to implement user help throu gh tip s or tac tics select ed and accumulated from a collection of common u ser- searc h mistakes. In such a case, the system would play a more activ e role by generatin g relevant search tips on th e fly and using zero-hits search resul ts as a basis for gener - ating a spe ll check or sugg esting altern ate wording. An idea l scenario is th at OPAC allow s the user to pursue mu ltiple avenues of an inq uiry by entering fra g- ments of th e question, exploring vocabulary choices, and reformulating the search wi th the assis tan ce of var iou s spec ialized intelligent assistants. Borgman suggests that an OPAC should be jud ge d by whether the ca tal og answers questions rather th an merely mat ches queri es. She s ugges ts the need to design systems that are ba sed on behavioral models of h ow people ask questions, argu- ing th a t users still need to tra n sla te their question into what a sys tem will accept. " User Instruction On- site tr aini ng and online documentation can help mak e it eas ier to u se OPAC. With the adven t of information lit- eracy, the shi ft in librar y instruction from procedur e- based query formulation to question-being-answered has taken place. At CSULA, in struct ion for en try-level classes focu ses on formulating a research sta teme nt and then identifying keywords and alternate terms. The instruc - tion sess ions that follow the initia l-conce pt formulation are sh ort an d focus on how to en ter keyword or subject, a u t h 01~ a n d title, and th e u se of Boolea n operators. Thi s approac h may improve success until th e sys tems provid e th e tools to improve sea rch stra tegies or accept an unt rai ned user 's input. As an increas ing numb er of users access online librar y ca talogs remotel y, assistance needs to be embedded into intuitive sys tems. "Time invested in elaborate help sys- tems often is better spent in redesigning the user interfac e so that help is no longer n eeded." 74 Users are not willing to devote much of their time to learning to use these sys- tems. They just want to get th eir searc h results quickly and expec t the catalog to be easy to use w ith little or no tim e invested in learning th e sys tem. I Conclusion The em piri cal study repo rted in thi s paper indicates th at p rogress has been made in terms of increasing search suc- cess by improv ing the OPAC search int erfac e. The goal is to design Web-based OPAC systems for today's users who are like ly to bring a mental model of Web search engin es to the lib rary catalog. Web-b ased OPACs and Web search engi nes differ in terms of th eir sys tems and interfac e design. However, in most cases, these differences do not res ult in different sea rch charac teris tics by users. Researc h findings on the impact of Web searc h engines and u ser searc hing expectations and behavior should b e ade- quately utilized to guide the in terface design. Web users typically do n ot know how a search engine works. Therefore, fund amental fea tures in the desi gn of the n ext generation of th e OPAC in terface should includ e ch ang in g the search to allow natural-language searching wit h keyword search first, and focu s on meetin g th e quick-search need . Such a concep t-ba sed sea rch will allow u sers to enter natu ra l lan guage of their chos en top ic in the searc h bo x w hil e th e system maps the quer y to th e s tru cture and content of the database. Relevance feedb ack to allow the system to brin g back related page s, spe llin g correctio n, and relevan ce-ranked output remain key goals for future OPACs. References and Notes 1. Sharon Seymour, "On line Public-Access Catalog User Stud ies: A Revi ew of Research Methodologies, March 1986- November 1989," Library and Information Science Research 13 (1991): 89-102; Andrew Large and Ja mshid Beheshti , "OPACs: A Resear ch Review," Library and Information Science Research 19 (1997): 2, 111-33. 2. Ibid., 113-16. 3. Ibid., 116-20. 4. Thomas A. Peters et al.," An Introduct ion to the Special Sec- tion on Transaction-Log Analysis," Library Hi Tech 11(1993): 2, 37. 5. Thomas A. Peters, "The History and Developm ent of Transaction- Log Analysis," Library Hi Tech 11 (1993): 2, 56. 6. Pauline A. Cochrane an d Karen Markey, "Cata log Use Studies since th e Introdu ction of Onlin e Interactiv e Ca tal ogs: Impact on Design for Subj ec t Access, " in Redesign of Catalogs and Indexes for Improved Subject Access: Selected Papers of Pauline A. Cochrnne (Phoenix: Oryx , 1985), 159-84; Steve n A. Zink , "Moni- toring User Success th ro u gh Transac tion-Log Analysis: The WolfPAC Example," Reference Services Review 19 (Sprin g 1991): 449- 56; Michael K. Buckl and et al., "OAS IS: A Front End for Prototy ping Catalog Enhancements," Library Hi Tech 10 (1992): 7-22. 7. Thomas A. Peters, "When Smart People Fail: An Analysis of the Tra nsaction Log of an On line Public-Access Catalog," Journal of Academic Librarianship 15 (1989): 5, 267. 8. Michael K. Buckland et al., "OASIS," 7-22. 9. Deborah D. Blecic et al., "Using Transac tion-Lo g Ana lys is to Imp rove OPAC Retrieval Result s," College and Research Libraries (Jan. 1998): 48. 10. Peters, "Histo ry and Development of Transacti on-Log Analys is," 2, 52. 11. Cha rles R. Hildr eth , "The Use and Understanding of Key- word Searching in a Un iversity Online Catalog," Information Technology and Libraries 16 (1997): 6. 12. Ray R. Larson, "Th e Decline of Subject Searching: Long - Term Trends and Patt erns of Index Use in an Online Catalog," Journal of the American Society for Information Science and Technol- ogy 42 (1991): 3, 210. 178 INFORMATION TECHNOLOGY AND LIBRARIES I DECEMBER 2004 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 13. John E. Tolle and Sehchang Hah, "O nline Search Patterns: NLM CATLINE Database," Journal of the American Society for Information Science 36 (Mar. 1985): 82- 93. 14. Carol Weiss Moore, "User Reac tion to Online Catalog s: An Exploratory Study," College and Research Libraries 42 (1981): 295-302; Joseph R. Matth ews et a l., Using Online Catalogs: A Nationwide Survey-A Report of a Study Sponsored by the Council on Library Resources (New York: N ea l-Schuman, 1983), 144. 15. Rhonda N. Hunter, "Success and Failures of Patrons Searching the Online Catalog at a Large Academic Library: A Transaction-Log Analysis," R.Q 30 (Spring 1991): 399. 16. Noelle Van Puli s and Lorne E. Ludy, "Subject Searching in an Onl ine Cata log with Aut h ority Contro l," College and Research Libraries 49 (1988): 526. 17. Hildret h, "Th e Use and Understanding of Keyword Searching," 6. 18. Ray R. Larson, "The Decline of Subjec t Searching," 3, 60. 19. Ibid. 20. Van Pulis and Ludy, "Subj ect Searching in an Onlin e Cat - alog," 527. 21. Karen Markey, Research Report on the Process of Subject Searching in the Library Catalog: Final Report of the Subject Access Research Project (repo rt no. OCLC /OP R/ RR-83-1) (Dub lin , Ohio: OCLC Online Co mput er Library Center, 1983), 529. 22. Pe ters, "The History and Deve lopment o f Transaction- Log Ana lysis," 2, 43. 23. Hi ldr eth, "The Use and Understanding of Keyword Searching," 8-9. 24. David R. Gerhan, "LCSH in vivo: Subje ct Searching Per- formance an d Strategy in th e OPAC Era," Journal of Academic Librarianship 15 (1989): 86-8 7. 25. Joan M. Cherry, "Improving Subject Access in OP ACS: An Exploratory Study of Conversion of Users' Queries," Journal of Academic Librarianship 18 (1992): 2, 98. 26. Rosemary Thorne and Jo Bell Whitlatch, "Patron On line Catalog Success," College and Research Libraries 55 (1994): 496. 27. Peters, "The History and Developmen t of Transaction- Log Analys is," 2, 48. 28. Ibid. 29. H unt er, "Succe ss and Failures," 400. 30. Ibid., 399. 31. Ibid., 400. 32. Peters, "The Histor y and Developmen t of Transa ction- Log Analysis," 2, 56. 33. Hunter, "Success and Failures," 400. 34. Hildreth , "The Use and Understandi n g of Keyword Searchi n g," 6. 35. Patricia M . Wa llace, "How Do Patrons Search th e Online C:, talog W h en No One 's Looking? Trnn sae tion-Log A nal ysis and Impli cation s for Bibliographic Instruction and System Desi gn, " RQ 33 (winter 1993): 3, 249. 36. Large and Beheshti, "OPACs: A Research Review," 125. 37. M. M. Hancock-Beaulieu , "Online Cata logue: A Case for the User," in The Online Catalogue: Developments and Directions, C. Hildreth, ed. (London: Library Association , 1989), 25-46. 38. Terry Ballard, "Com parative Searching Styles of Patrons and Staff," Library Resources and Technical Services 38 (1994): 293- 305. 39. Jane Scott et al.,"@*&#@ This Computer and the Horse It Rode in On: Patron Frustration and Failur e at th e OPAC" (in "Co ntinuity and Transformation : The Promise of Confluen ce": U SABi Li rs·"' I [: ,, ), B p l..JR l.i ""( ' " User Interface Consulting FED ERAT ED SEARCH tN(,lN ES 1.IBR;'.'\RY PORTALS & [)AT/\, (LN 'ITR S ()PACS f.." ( H i LDREi' l's Dl(, ITAL LIBR AR IES Ezra Schwartz LOCS (773) 256-1418 ezra@artandtech.com http://www.artandtech.com Proceedings of the ACRL 7th Nationa l Conference, Chicago: ACRL 1995), 247-56. 40. Thorne and Whitlat ch, "Patron On lin e Catalog Success," 496. 41. Blecic et al., "Usin g Tran sac tion-Log Ana lys is," 46. 42. Virginia Ortiz-Repiso and Purificac ion Moscoso, "We b- Based OP A Cs: Between Tradition and Innovation ," lnformntion Technology and Libraries 18, no. 2 (June 1999): 68-69. 43. Hildreth, "The Use and Understanding of Keyword Search- ing," 6. 44. Ortiz-Repiso and Mos coso, "Web-Bas ed OPAC s," 71. 45. Ibid., 75. 46. Chris tine Borgm an, "Why Are On line Catalogs Still Hard to Us e?" Journal of the Americnn Society for Information Science 47 (1996): 7, 501. 47. Van Pulis and Ludy, "Subje ct Searching in an Onlin e Cat - alog," 53. 48. Rla zek and Bilal , "Prob lems with OPAC: A Case Study of an Academic Research Library," RQ 28 (w int er 1988): 175. 49. Debora h D. Blecic et al., "A Longitud inal Stu dy of the Effects of OPAC Screen Changes on Searching Behavior and User Success," College and Research Library 60, no. 6 (Nov. 1999): 524,527. 50. Bernar d J. Jan sen and Udo Pooch, "A Revi ew of Web Searching Studies and a Framework for Future Resear ch," jour- nal of the American Society for Information Science and Technology 52 (2001): 3, 249-50. 51. Ibid., 250. 52. Blazek and Bilal, "Problems with OPAC: A Case Study," 175; Moore , "User Reaction to Online Cata logs," 295-302. THE IMPACT OF WEB SEARCH ENGINES ON SUBJECT SEARCHING IN OPAC I YU AND YOUNG 179 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 53. M. J. Bates, "The Design of Browsin g and Berry-Pickin g Techniques for the Onlin e Search Interfac e," Online Review 13 (1989): 5, 407-24. 54. Jan sen and Pooch, "A Review of Web Searc hing Studies, " 238. 55. Judy Luther, "Trumping Google? Metasearching's Promise," Library Journal 128 (2003): 16, 36. 56. Jack Muramatsu and Wanda Pratt, "Transparent Queries: Investigating Users' Mental Models of Search Engines," Research and Development in Information Retrieval Sept. 2001. Accessed Mar. 10, 2003, http://citeseer.nj.nec.com/muramatsuOltransparent. html. 57. Jans en and Pooch, "A Review of Web Searching Studie s," 235. 58. Luth er, 'T rumping Goog le," 36. 59. Blecic et a l., "A Lon gitudina l Study of th e Effects of OPAC Screen Changes," 527. 60. Sus an M. Colaric, "Ins truction for Web Searching: An Empirical Study," College and Research Libraries News, 64 (2003): 2. 61. A. G. Sutcliff, M. Ennis, and S. J. Watkinson, "Empirical Studies of End-User Informati on Searching," Joumal of the Ameri- can Society for Information Science and Tcchnologtj 51 (2000): 13, 1213. 62. "A ll About Google," Google. Accessed Dec. 10, 2003, www.google.com. 63. G. Salton, Introduction to Modern Information Retrieval (New York: McGraw-Hill, 1983), 18. 64. Orti z-Rep iso and Moscoso, "We b-Ba sed OPACs," 71. 65. Luth er, "Trumping Google," 37. 66. Maaike D. Kiestr a et al, "End-Us ers Searching th e Online Catalogue: The Influenc e of Domain and System Knowledge on Search Patterns. Experiment at Tilburg University," The Elec- tronic Library 12 (Dec. 1994): 335-43. 67. C. Jen kins et al., "Pa tterns of In forma tion Seeking on the Web: A Qualitative Study of Domain Expertise and Web Experti se," IT and Society l (Winter 2003): 3, 74,77. Accessed May 10, 2003, www.ItandSociety.org/. 68. C. Holscher and G. Strube, "Web Search Behavior of Inter- net Experts and Newbi es," 9th International World Wide Web Con- ference, (Amsterdam. 2000). Accessed Mar. 28, 2003, www9.org/ w9cdrom /8 1/81.html; A. Pollock and A. Hockley, "Wha t's Wrong with Internet Searching," D-lib Magazine (Mar. 1997). Accessed May 10, 2003, www.dlib.org/dlib/march97 /b t /03 pollo ck.h tml. 69. M . M . Hanco ck-Beau lieu , "On lin e Catalogue: A Case for the User," 25-46. 70. Wilbert 0. Galitz, The Essential Guide to User Interface Design: An Introduction to GUI Design Principles and Techniques (Chichester, England: Wiley, 1996). 71. Juliana Chan," An Evaluation of Displays of Bibliographic Records in OPACs in Canadian Academic and Public Libraries," MIS Report, Univ. of Toronto, 1995. [025.3132 C454E] 72. Giorgio Brajnik et al., "Strategic H elp in User Interfaces for Information Retriev al," Journal of the American Society for Information Science and Technology (JASIST) 53 (2002): 5, 344 . 73. Borgman, "Why Are Online Catalogs Still Hard to Use?" 500. 74. Ibid . 180 INFORMATION TECHNOLOGY AND LIBRARIES I DECEMBER 2004