214 iNFOrMAtiON tecHNOlOGY AND liBrAries | DeceMBer 2010 Margaret Brown-Sica, Jeffrey Beall, and Nina McHale Next-Generation Library Catalogs and the Problem of Slow Response Time and librarians will benefit from knowing what typical and acceptable response times are in online catalogs, and this information will assist in the design and evaluation of library discovery systems. This study also looks at bench- marks in response time and defines what is unacceptable and why. When advanced features and content in library catalogs increase response time to the extent that users become disaffected and use the catalog less, NextGen cata- logs represent a step backward, not forward. In August 2009, the Auraria Library launched an instance of the WorldCat Local product from OCLC, dubbed WorldCat@Auraria. The Library’s traditional catalog—named Skyline and running on the Innovative Interfaces platform—still runs concurrently with WorldCat@Auraria. Because WorldCat Local currently lacks a library circulation module that the Library was able to use, the legacy catalog is still required for its circulation functionality. In addition, Skyline contains MARC records from the SerialsSolution 360 MARC prod- uct. Since many of these records are not yet available in the OCLC WorldCat database, these records are being maintained in the legacy catalog to enable access to the Library’s extensive collection of online journals. Almost immediately upon implementation of WorldCat Local, many Library staff began to express concern about the product’s slow response time. They bemoaned its slowness both at the reference desk and during library instruction sessions. Few of the discus- sions of the product’s slow response time evaluated this weakness in the context of its advanced features. Several of the reference and instruction librarians even stated that they refused to use it any longer and that they were not recommending it to students and faculty. Indeed, many stated that they would only use the legacy Skyline catalog from then on. Therefore we decided to analyze the product’s response time in relation to the legacy catalog. We also decided to further our study by examining response time in library catalogs in general, including several different online catalog products from different vendors. ■■ Response Time The term response time can mean different things in dif- ferent contexts. Here we use it to mean the time it takes for all files that constitute a single webpage (in the case of testing performed, a permalink to a bibliographic record) to travel across the Internet from a Web server to the computer on which the page is to be displayed. We do not include the time it takes for the browser to render the page, only the time it takes for the files to arrive to the requesting computer. Typically, a single webpage is made of multiple files; these are sent via the Internet from a Web Response time as defined for this study is the time that it takes for all files that constitute a single webpage to travel across the Internet from a Web server to the end user’s browser. In this study, the authors tested response times on queries for identical items in five different library catalogs, one of them a next-generation (NextGen) catalog. The authors also discuss acceptable response time and how it may affect the discovery process. They suggest that librarians and vendors should develop standards for acceptable response time and use it in the product selec- tion and development processes. N ext-generation, or NextGen, library catalogs offer advanced features and functionality that facilitate library research and enable Web 2.0 features such as tagging and the ability for end users to create lists and add book reviews. In addition, individual catalog records now typically contain much more data than they did in earlier generations of online catalogs. This additional data can include the previously mentioned tags, lists, and reviews, but a bibliographic record may also con- tain cover images, multiple icons and graphics, tables of contents, holdings data, links to similar items, and much more. This additional data is designed to assist catalog users in the selection, evaluation, and access of library materials. However, all of the additional data and features have the disadvantage of increasing the time it takes for the information to flow across the Internet and reach the end user. Moreover, the code that handles all this data is much more complex than the coding used in earlier, traditional library catalogs. Slow response time has the potential to discourage both library patrons from using the catalog and library staff from using or recommending it. During a reference interview or library instruction ses- sion, a slow response time creates an awkward lull in the process, a delay that decreases confidence in the mind of library users, especially novices who are accustomed to the speed of an open Internet search. The two-fold purpose of this study is to define the concept of response time as it relates to both traditional and NextGen library catalogs and to measure some typical response times in a selection of library catalogs. Libraries Margaret Brown-sica (margaret.brown-sica@ucdenver.edu) is assistant Professor, associate Director of Technology Strat- egy and learning Spaces, Jeffrey Beall (jeffrey.beall@ucden- ver.edu) is assistant Professor, Metadata librarian, and Nina McHale (nina.mchale@ucdenver.edu) is assistant Professor, web librarian, university of colorado Denver. Next-GeNerAtiON liBrArY cAtAlOGs | BrOWN-sicA, BeAll, AND McHAle 215 Mathews posted an article called “5 Next Gen Library Catalogs and 5 Students: Their Initial Impressions.”7 Here he shares student impressions of several NextGen catalogs. Regarding slow response time Mathews notes, “Lots of comments on slowness. One student said it took more than ten seconds to provide results. Some other comments were: ‘that’s unacceptable’ and ‘slow-motion search, typical library.’” Nagy and Garrison, on Lauren’s Library Blog, emphasized that any “cross-silo federated search” is “as slow as the slower silos.”8 Any search inter- face is as slow as the slowest database from which it pulls information; however, that does not make users more likely to wait for search results. In fact, many users will not even know—or care—what is happening behind the scenes in a NextGen catalog. The assertion that slow response time makes well- intentioned improvements to an interface irrelevant is supported by an article that analyzes the development of Semantic Web browsers. Frachtenberg notes that users, however, have grown to expect Web search engines to provide near-instantaneous results, and a slow search engine could be deemed unusable even if it provides highly relevant results. It is therefore imperative for any search engine to meet its users’ interactivity expectations, or risk losing them.9 This is not just a library issue. Users expect a fast response to all Web queries, and we can learn from studies on general Web response time and how it affects the user experience. Huang and Fong-Ling help explain different user standards when using websites. Their research suggests that “hygiene factors” such as “navigation, information display, ease of learning and response time” are more important to people using “utilitarian” sites to accomplish tasks rather than “hedo- nistic” sites.10 In other words, response time importance increases when the user is trying to perform a task— such as research—and possibly even more for a task that may be time sensitive—such as trying to complete an assignment for class. ■■ Method For testing response time in an assortment of library cat- alogs, we used the WebSitePulse service (http://www .websitepulse.com). WebSitePulse provides in-depth website and server diagnostic services that are intended to save e-business customers time and money by reporting errors and Web server and website performance issues to clients. A thirty-day free trial is available for potential cus- tomers to review the full array of their services; however, the free Web Page Test, available at http://www.website server and arrive sequentially at the computer where the request was initiated. While the World Wide Web Consortium (W3C) does not set forth any particular guidelines regarding response time, go-to usability expert Jakob Nielsen states that “0.1 second is about the limit for having the user feel that the system is reacting instantaneously.”1 He further posits that 1.0 second is “about the limit for the user’s flow of thought to stay uninterrupted, even though the user will notice the delay.”2 Finally, he asserts that: 10 seconds is about the limit for keeping the user’s attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially impor- tant if the response time is likely to be highly variable, since users will then not know what to expect.3 Even though this advice dates to 1994, Nielsen noted even then that it had “been about the same for many years.”4 ■■ Previous Studies The chief benefit of studying response time is to estab- lish it as a criterion for evaluating online products that libraries license and purchase, including NextGen online catalogs. Establishing response-time benchmarks will aid in the evaluation of these products and will help libraries convey the message to product vendors that fast response time is a valuable product feature. Long response times will indicate that a product is deficient and suffers from poor usability. It is important to note, however, that sometimes library technology environments can be at fault in lengthening response time as well; in “Playing Tag In the Dark: Diagnosing Slowness In Library Response Time,” Brown-Sica diag- nosed delays in response time by testing such variables as vendor and proxy issues, hardware, bandwidth, and network traffic.5 In that case, inadequate server specifi- cations and settings were at fault. While there are many articles on NextGen catalogs, few of them discuss the issue of response time in rela- tion to their success. Search slowness has been reported in library literature about NextGen catalogs’ metasearch cousins, federated search products. In a 2006 review of federated search tools MetaLib and WebFeat, Chen noted that “a federated search could be dozens of times slower than Google.”6 More comments about the negative effects of slow response time in NextGen catalogs can be found in popular library technology blogs. On his blog, 216 iNFOrMAtiON tecHNOlOGY AND liBrAries | DeceMBer 2010 ■■ Findings: Skyline Versus WorldCat@Auraria In figure 2, the bar graph shows a sample load time for the permalink to the bibliographic record for the title Hard Lessons: The Iraq Reconstruction Experience in Skyline, Auraria’s traditional catalog load time for the page is pulse.com/corporate/alltools.php, met our needs. To use the webpage test, simply select “Web Page Test” from the dropdown menu, input a URL—in the case of the testing done for this study, the perma- link for one of three books (see, for example, figure 1)—enter the validation code, and click “Test It.” WebSitePulse returns a bar graph (figure 2) and a table (figure 3) of the file activity from the server sending the composite files to the end user ’s Web browser. Each line represents one of the files that make up the rendered webpage. They load sequentially, and the bar graph shows both the time it took for each file to load and the order in which the files were received. Longer seg- ments of the bar graph provide visual indication of where a slow-loading webpage might encounter sticking points—for example, wait- ing for a large image file or third-party content to load. Accompanying the bar graph is a table describing the file transmissions in more detail, including DNS, connection, file redirects (if applicable), first and last bytes, file trans- mission times, and file sizes. Figure 1. Permalink screen shot for the record for the title Hard Lessons in Auraria Library’s Skyline catalog Figure 2. WebSitePulse webpage test bar graph results for Skyline (traditional) catalog record Figure 3. WebSitePulse webpage test table results for Skyline (traditional) catalog record Next-GeNerAtiON liBrArY cAtAlOGs | BrOWN-sicA, BeAll, AND McHAle 217 requested at items 8, 14, 15, 17, 26, and 27. The third parties include Yahoo! API services, the Google API ser- vice, ReCAPTCHA, and AddThis. ReCAPTCHA is used to provide security within WorldCat Local with opti- cal character recognition images (“captchas”), and the AddThis API is used to provide bookmarking function- ality. At number 22, a connection is made to the Auraria Library Web server to retrieve a logo image hosted on the Web server. At number 28, the cover photo for Hard Lessons is retrieved from an OCLC server. The files listed in figure 6 details the complex process of Web brows- ers’ assembly of them. Each connection to third-party content, while all relatively short, allows for addi- tional features and functionality, but lengthens overall response. As figure 6 shows, the response time is slightly more than 10 seconds, which, according to Nielsen, “is about the limit for keeping the user ’s attention focused on the dialogue.”12 While widgets, third-party content, and other Web 2.0 tools add desirable content and functionality to the Library’s catalog, they also do slow response time considerably. The total file size for the bibliographic record in WorldCat@Auraria—compared to Skyline’s 84.64 KB—is 633.09 KB. As will be shown in the test results below for the catalog and NextGen catalog products, bells and whistles added to traditional 1.1429 seconds total. The record is composed of a total of fourteen items, including image files (GIFs), cascad- ing style sheet (CSS) files, and JavaScript (JS) files. As the graph is read downward, the longer segments of the bars reveal the sticking points. In the case of Skyline, the nine image files, two CSS files, and one JS file loaded quickly; the only cause for concern is the red line at item four. This revealed that we were not taking advantage of the option to add a favicon to our III catalog. The Web librarian provided the ILS server technician with the same favi- con image used for the Library’s website, correcting this issue. The Skyline catalog, judging by this data, falls into Nielsen’s second range of user expectations regarding response time, which is more than one second, or “about the limit for the user’s flow of thought to stay uninter- rupted, even though the user will notice the delay.”11 Further detail is provided in figure 3; this table lists each of the webpage’s component files, and various times asso- ciated with the delivery of each file. The column on the right lists the size in kilobytes of each file. The total size of the combined files is 84.64 KB. In contrast to Skyline’s meager 14 files, WorldCat Local requires 31 items to assemble the webpage (figure 4) for the same bibliographic record. Figures 5 and 6 show that this includes 10 CSS files, 10 JavaScript files, and 8 images files (GIFs and PNGs). No item in particular slows down the overall process very much; the longest- loading item is number 13, which is a wait for third-party content, a connection to Yahoo!’s User Interface (YUI) API service. Additional third-party content is being Figure4. Permalink screen shot for the record for the title Hard Lessons in WorldCat@Auraria Figure 5. WebSitePulse webpage test bar graph results for WorldCat@Auraria record 218 iNFOrMAtiON tecHNOlOGY AND liBrAries | DeceMBer 2010 total time for each permalinked bibliographic record to load as reported by the WebSitePulse tests; this number appears near the lower right-hand corner of the tables in figures 3, 6, 9, 12, and 15. We selected three books that were each held by all five of our test sites, verifying that we were search- ing the same three bibliographic records in each of the online catalogs by looking at the OCLC number in the records. Each of the catalogs we tested has a permalink feature; this is a stable URL that always points to the same record in each catalog. Using a permalink approximates conducting a known-item search for that item from a catalog search screen. We saved these links and used them in our searches. The bib- liographic records we tested were for these books; the permalinks used for testing follow the books: Book 1: Hard Lessons: The Iraq Reconstruction Experience. Washington, D.C.: Special Inspector General, Iraq Reconstruction, 2009 (OCLC number 302189848). Permalinks used: ■■ WorldCat@Auraria: http://aurarialibrary.worldcat .org/oclc/302189848 ■■ Skyline: http://skyline.cudenver.edu/record=b243 3301~S0 ■■ LCOC: http://lccn.loc.gov/2009366172 ■■ UT Austin: http://catalog.lib.utexas.edu/record= b7195737~S29 ■■ USC: http://library.usc.edu/uhtbin/cgisirsi/ x/0/0/5?searchdata1=2770895{CKEY} Book 2: Ehrenreich, Barbara. Nickel and Dimed: On (Not) Getting by in America. 1st ed. New York: Metropolitan, 2001 (OCLC number 256770509). Permalinks used: ■■ WorldCat@Auraria: http://aurarialibrary.worldcat .org/oclc/45243324 ■■ Skyline: http://skyline.cudenver.edu/record=b187 0305~S0 ■■ LCOC: http://lccn.loc.gov/00052514 ■■ UT Austin: http://catalog.lib.utexas.edu/record= b5133603~S29 ■■ USC: http://library.usc.edu/uhtbin/cgisirsi/ x/0/0/5?searchdata1=1856407{CKEY} Book 3: Langley, Lester D. Simón Bolívar: Venezuelan Rebel, American Revolutionary. Lanham: Rowman & Littlefield catalogs slowed response time considerably, even dou- bling it in one case. Are they worth it? The response of Auraria’s reference and instruction staff seems to indi- cate that they are not. ■■ Gathering More Data: Selecting the Books and Catalogs to Study To broaden our comparison and to increase our data collection, we also tested three other non-Auraria cata- logs. We designed our study to incorporate a number of variables. We decided to link to bibliographic records for three different books in the five different online catalogs tested. These included Skyline and WorldCat@Auraria as well three additional online public access catalog products, for a total of two instances of Innovative Interfaces products, one of a Voyager catalog, and one of a SirsiDynix catalog. We also selected online catalogs in different parts of the country: WorldCatLocal in Ohio; Skyline in Denver; the Library of Congress’ Online Catalog (LCOC) in Washington, D.C.; the University of Texas at Austin’s (UT Austin) online catalog; and the University of Southern California’s (USC) online catalog, named Homer, in Los Angeles. We also did our testing at different times of the day. One book was tested in the morning, one at midday, and one in the afternoon. WebSitePulse performs its webpage tests from three different locations in Seattle, Munich, and Brisbane; we selected Seattle for all of our tests. We recorded the Figure 6. WebSitePulse webpage test table results for WorldCat@Auraria record Next-GeNerAtiON liBrArY cAtAlOGs | BrOWN-sicA, BeAll, AND McHAle 219 .org/oclc/256770509 ■■ Skyline: http://skyline.cudenver.edu/record=b242 6349~S0 ■■ LCOC: http://lccn.loc.gov/2008041868 ■■ UT Austin: http://catalog.lib.utexas.edu/record= b7192968~S29 ■■ USC: http://library.usc.edu/uhtbin/cgisirsi/ x/0/0/5?searchdata1=2755357{CKEY} We gathered the data for thirteen days in early November 2009, an active period in the middle of the semester. For each test, we recorded the response time total in seconds. The data is displayed in tables 1–3. We searched bibliographic records for three books in five library catalogs over thirteen days (3 x 5 x 13) for a total of 195 response time measurements. The WebSitePulse data is calculated to the ten thousandth of a second, and we recorded the data exactly as it was presented. Publishers, c2009 (OCLC number 256770509). Permalinks used: ■■ WorldCat@Auraria: http://aurarialibrary.worldcat Table 1. Response Times for Book 1 Response time in seconds Day Wor ld- Cat Skyline LC UT Austin USC 1 10.5230 1.3191 2.6366 3.6643 3.1816 2 10.5329 1.2058 1.2588 3.5089 4.0855 3 10.4948 1.2796 2.5506 3.4462 2.8584 4 13.2433 1.4668 1.4071 3.6368 3.2750 5 10.5834 1.3763 3.6363 3.3143 4.6205 6 11.2617 1.2461 2.3836 3.4764 2.9421 7 20.5529 1.2791 3.3990 3.4349 3.2563 8 12.6071 1.3172 3.6494 3.5085 2.7958 9 10.4936 1.1767 2.6883 3.7392 4.0548 10 10.1173 1.5679 1.3661 3.7634 3.1165 11 9.4755 1.1872 1.3535 3.4504 3.3764 12 12.1935 1.3467 4.7499 3.2683 3.4529 13 11.7236 1.2754 1.5569 3.1250 3.1230 Average 11.8310 1.3111 2.5105 3.4874 3.3953 Table 2. Response Times for Book 2 Response time in seconds Day World- Cat Skyline LC UT Austin USC 1 10.9524 1.4504 2.5669 3.4649 3.2345 2 10.5885 1.2890 2.7130 3.8244 3.7859 3 10.9267 1.3051 0.2168 4.0154 3.6989 4 13.8776 1.3052 1.3149 4.0293 3.3358 5 10.6495 1.3250 4.5732 3.5775 3.2979 6 11.8369 1.3645 1.3605 3.3152 2.9023 7 11.3482 1.2348 2.3685 3.4073 3.5559 8 10.7717 1.2317 1.3196 3.5326 3.3657 9 11.1694 1.0997 1.0433 2.8096 2.6839 10 19.0694 1.6479 2.5779 4.3595 2.6945 11 12.0109 1.1945 2.5344 3.0848 18.5552 12 12.6881 0.7384 1.3863 3.7873 3.9975 13 11.6370 1.1668 1.2573 3.3211 3.6393 Average 12.1174 1.2579 1.9410 3.5791 4.5190 Table 3. Response Times for Book 3 Response time in seconds Day World- Cat Skyline LC UT Austin USC 1 10.8560 1.3345 1.9055 3.7001 2.6903 2 10.1936 1.2671 1.8801 3.5036 2.7641 3 11.0900 1.5326 1.3983 3.5983 3.0025 4 10.9030 1.4557 2.0432 3.6248 2.9285 5 12.3503 1.5972 3.5474 3.6428 4.5431 6 9.1008 1.1661 1.4440 3.4577 3.1080 7 9.6263 1.1240 2.3688 3.1041 3.3388 8 10.9539 1.1944 1.4941 2.8968 3.4224 9 11.0001 1.2805 1.3255 3.3644 2.7236 10 10.2231 1.3778 1.3131 3.3863 3.4885 11 10.1358 1.2476 2.3199 3.4552 2.9302 12 12.0109 1.1945 2.5344 3.0848 18.5552 13 11.5881 1.2596 2.5245 3.8040 3.8506 Average 10.7717 1.3101 2.0076 3.4325 4.4112 Table 4. Averages Response time in seconds Book World- Cat Skyline LC UT Austin USC Book 1 11.8310 1.3111 2.5105 3.4874 3.3953 Book 2 12.1174 1.2579 1.9410 3.5791 4.5190 Book 3 10.7717 1.3101 2.0076 3.4325 4.4112 Average 11.5734 1.2930 2.1530 3.4997 4.1085 220 iNFOrMAtiON tecHNOlOGY AND liBrAries | DeceMBer 2010 university of colorado Denver: skyline (innovative interfaces) As previously mentioned, the traditional catalog at Auraria Library runs on an Innovative Interfaces integrated library system (ILS). Testing revealed a missing favicon image file that the Web server tries to send each time (item 4 in figure 3). However, this did not negatively affect the response time. The catalog’s response time was good, with an aver- age of 1.2930 seconds, giving it the fastest average time among all the test sites in the testing period. As figure 1 shows, however, Skyline is a typical legacy catalog that is designed for a traditional library environment. library of congress: Online catalog (voyager) The average response time for the LCOC was 2.0076 ■■ Results The data shows the response times for each of the three books in each of the five online catalogs over the thirteen- day testing period. The raw data was used to calculate averages for each book in each of the five online catalogs, and then we calculated averages for each of the five online catalogs (table 4). The averages show that during the testing period, the response time varied between 1.2930 seconds for the Skyline library catalog in Denver to 11.5734 seconds for WorldCat@Auraria, which has its servers in Ohio. university of colorado Denver: Worldcat@Auraria WorldCat@Auraria was routinely over Nielsen’s ten- second limit, sometimes taking as long as twenty sec- onds to load all the files to generate a single webpage. As previously discussed, this is due to the high number and variety of files that make up a single bibliographic record. The files sent also include cover images, but they are small and do not add much to the total time. After our tests on WorldCat@Auraria were conducted, the site removed one of the features on pages for individual resources, namely the “similar items” feature. This fea- ture was one of the most file-intensive on a typical page, and its removal should speed up page loads. However, WorldCat@Auraria had the highest average response time by far of the five catalogs tested. Figure 7. Permalink screen shot for the record for the title Hard Lessons in the Library of Congress online catalog Figure 8. WebSitePulse webpage test bar graph results for Library of Congress online catalog record Figure 9. WebSitePulse webpage test table results for Library of Congress online catalog record Next-GeNerAtiON liBrArY cAtAlOGs | BrOWN-sicA, BeAll, AND McHAle 221 item 14 is a script, that while hosted on the ILS server, que- ries Amazon.com to return cover image art (figures 11–12). The average response time for UT Austin’s catalog was 3.4997 seconds. This example demonstrates that response times for traditional (i.e., not NextGen) catalogs can be slowed down by additional content as well. university of southern california: Homer (sirsiDynix) The average response time for USC’s Homer catalog was 4.1085 seconds, making it the second slowest after seconds. This was the second fastest average among the five catalogs tested. While, like Skyline, the bibliographic record page is sparsely decorated (figure 7), this pays dividends in response time, as there are only two CSS files and three GIF files to load after the HTML content loads (figure 9). Figure 8 shows that initial connection time is the longest factor in load time; however, it is still short enough to not have a negative effect. Total file size is 19.27 KB. As with Skyline, the page itself (figure 7) is not particularly end-user friendly to nonlibrarians. university of texas at Austin: library catalog (innovative interfaces) UT Austin, like Auraria Library, runs an Innovative Interfaces ILS. The library catalog also includes book cover images, one of the most attractive NextGen features (figure 10), and as shown in figure 12, third-party content is used to add features and functionality (items 16 and 17). UT Austin’s catalog uses a Google JavaScript API (item 16 in figure 12) and LibraryThing’s Catalog Enhancement prod- uct, which can add book recommendations, tag browsing, and alternate editions and translations. Total content size for the bibliographic record is considerably larger than Skyline and the LCOC at 138.84 KB. It appears as though inclusion of cover art nearly doubles the response time; Figure 10. Permalink screen shot for the record for the title Hard Lessons in University of Texas at Austin’s library catalog Figure 11. WebSitePulse webpage test bar graph results for University of Texas at Austin’s library catalog record Figure 12. WebSitePulse webpage test table results for University of Texas at Austin’s library catalog record 222 iNFOrMAtiON tecHNOlOGY AND liBrAries | DeceMBer 2010 completed. Added functionality and features in library search tools are valuable, but there is a tipping point when these features slow down a product’s response time to where users find the search tool too slow or unreliable. Based on the findings of this study, we recom- mend that libraries adopt Web response time standards, such as those set forth by Nielsen, for evaluating vendor search products and creating in-house search products. Commercial tools like WebSitePulse make this type of data collection simple and easy. Testing should be con- ducted for an extended period of time, preferably during a peak period—i.e., during a busy time of the semes- ter for academic libraries. We further recommend that reviews of electronic resources add response time as an WorldCat@Auraria, and the slowest among the tradi- tional catalogs. This SirsiDynix catalog appears to take a longer time than the other brands of catalogs to make the initial connection to the ILS; this accounts for much of the slowness (see figures 14 and 15). Once the initial connection is made, however, the remaining content loads very quickly, with one exception: item 13 (see fig- ure 15), which is a connection to the third-party provider Syndetic Solutions, which provides cover art, a summary, an author biography, and a table of contents. While the display of this content is attractive and well-integrated to the catalog (figure 13), it adds 1.2 seconds to the total response time. Also, as shown in item 14 and 15, USC’s Homer uses the AddThis service to add bookmarking enhancements to the catalog. Total combined file size is 148.47 KB, with the bulk of the file size (80 KB) coming from the initial connection (item 1 in figure 15). ■■ Conclusion An eye-catching interface and valuable content are lost on the end user if he or she moves on before a search is Figure 13. Permalink screen shot for the record for the title Hard Lessons in Homer, the University of Southern California’s catalog Figure 14. WebSitePulse webpage test bar graph results for Homer, the University of Southern California’s catalog Figure 15. WebSitePulse webpage test table results for Homer, the University of Southern California’s catalog Next-GeNerAtiON liBrArY cAtAlOGs | BrOWN-sicA, BeAll, AND McHAle 223 4. Ibid. 5. Margaret Brown-Sica. “Playing Tag In the Dark: Diagnos- ing Slowness In Library Response Time,” Information Technology & Libraries 27, no. 4 (2008): 29–32. 6. Xiaotian Chen, “MetaLib, WebFeat, and Google: The Strengths and Weaknesses of Federated Search Engines Com- pared with Google,” Online Information Review 30, no. 4 (2006): 422. 7. Brian Mathews, “5 Next Gen Library Catalogs and 5 Stu- dents: Their Initial Impressions,” online posting, May 1, 2009, The Ubiquitous Librarian Blog, http://theubiquitouslibrarian .typepad.com/the_ubiquitous_librarian/2009/05/5-next-gen- library-catalogs-and-5-students-their-initial-impressions.html (accessed Feb. 5, 2010) 8. Andrew Nagy and Scott Garrison, “Next-Gen Catalogs Are Only Part of the Solution,” online posting. Oct. 4, 2009, Lauren’s Library Blog, http://laurenpressley.com/library/2009/10/next -gen-catalogs-are-only-part-of-the-solution/ (accessed Feb. 5, 2010). 9. Eitan Frachtenberg, “Reducing Query Latencies in Web Search Using Fine-Grained Parallelism,” World Wide Web 12, no. 4 (2009): 441–60. 10. Travis K Huang and Fu Fong-Ling, “Understanding User Interface Needs of E-commerce Web Sites,” Behaviour & Information Technology 28, no. 5 (2009): 461–69, http://www .informaworld.com/10.1080/01449290903121378 (accessed Feb. 5, 2010). 11. Nielsen, Usability Engineering, 135. 12. Ibid. evaluation criterion. Additional research about response time as defined in this study might look at other search tools, to include article databases, and especially other metasearch products that collect and aggregate search results from several remote sources. Further studies with more of a technological focus could include discussions of optimizing data delivery methods—again, in the case of metasearch tools from multiple remote sources—to reduce response time. Finally, product designers should pay close attention to response time when designing information retrieval products that libraries purchase. ■■ Acknowledgments The authors wish to thank Shelley Wendt, library data analyst, for her assistance in preparing the test data. References 1. Jakob Nielsen, Usability Engineering (San Francisco: Morgan Kaufmann, 1994): 135. 2. Ibid. 3. Ibid.