College and Research Libraries ROBERT BALAY and CHRISTINE ANDREW Use of the Reference Service in a Large Academic Library A method for collecting and analyzing records of reference inquiries received by the Reference Department of Yale University Library is described. Variables observed included day, hour, patron affiliation, inquiry type, inquiry mode, time expended, and search points. In- quiries were recorded on forms which also functioned as worksheets for keypunching. An existing computer program generated tables in which observed variables were compared. Tables of search units were also prepared. Conclusions were drawn regarding traffic, staffing, out- side use of the library, need for user training programs, and the like .. Methods were found to be simple and economical, and information useful in the management of reference service was derived. THis PAPER SUMMARIZES A STUDY of the use of reference service at Sterling Me- morial Library, the central research li- brary of Yale University. The study was conducted during three weeks of April 1970, one week of November 1970, one week of January 1971, and two weeks of April1971. The study grew out of the dissatisfac- tion which the staff of the library, s Ref- erence Department felt toward the methods of gathering reference statis- tics which had been used in the past. Reference questions had simply been recorded by entering marks on a sheet that had been divided into seven parts to represent the days of the week. The Robert Balay is head, Reference Depart- ment, Yale University Library. Christine Andrew is librarian, Bibliographic Center, Research Libraries Group, New Haven, Connecticut. sheets were collected at the end of the week, the numbers tallied, and the re- sults reported in the departmenf s an- nual report. The count was not taken during the entire year, but only for a two- or three-week period in April, at the end of the annual reporting period. This practice, or something like it, is commonly used in university libraries. Annual reports of these libraries often contain sections which tabulate the number of reference inquiries for the reporting year and compare this with the performance of prior years. The method is nevertheless manifest- ly unsatisfactory, as librarians acknowl- edge in writing about the measurement of reference service. A tone of pessi- mism pervades the literature on the sub- ject. Rogers speaks of "the follies we commit and the fallacies we perpetu- ate,, with regard to reference statistics, and Rothstein characterizes the attitude of those who have written about statis- I 9 10 I College & Research Libraries • January 1975 tics of reference service as one of "querulous diffidence."1• 2 A handbook on library statistics, whose aims were to standardize concepts, definitions, and terminology and to make recommenda- tions with regard to the reporting of li- brary statistics, advised flatly that so far as reference service is concerned, ''N a- tiona! data are not feasible at this time."3 To some extent, the pessimism is justified. Studies which have analyzed large numbers of inquiries have tended, like Guerrier, to consider only the kinds of questions being asked. 4 Occasionally, efforts have been made to learn some- thing about users as well as about ques- tions; often, however, the number of inquiries is so small as to cast doubt on the result, 5 or the information gathered about users is of interest to public li- braries but not to others. 6 Cole's study was carefully planned, but specifically excluded research libraries. 7 Lacking al- together has been a methodology for gathering detailed information about patterns of reference use-when vari- ous categories of users enter the library, what questions they ask, how the ques- tions asked by one category differ from those asked by another, what parts of the library's collections are used in an- swering various kinds of inquiries, and so on. One may well ask in light of this whether there is any profit in undertak- ing yet another statistical study of ref- erence use. It was our view that such a survey may be justified if it provides a means of evaluating the quality of ref- erence service being offered, or if it as- sists in the management of reference service by providing detailed informa- tion about patterns of use. The present study addressed itself primarily to the latter point. We also hoped to develop a methodology which could be adapted for use in other libraries and which could thereby contribute toward the de- velopment of standards of reference service. The principal aim, however, was to find out about patterns of reference use in our own library, an aim that we believed would justify the effort expend- ed. REQUIREMENTS Several factors governed the design of this study. 1. In order to provide a firm basis for generalizations about reference use,. a large number of observations. would be required. 2. In order to determine total use pat- terns, all questions would have to be recorded, no matter how trivial they might seem. 3. Since funds were limited, costs would have to be low. 4. Since we would have to use the working reference staff to collect data, procedures would have to be easy to administer and forms sim- ple to complete. 5. Data collecting would have to be done in such a way as to interfere as little as possible with normal reference work. 6. Data would have to be capable of being analyzed in such a way as to permit comparison of various items in it, so that variables could be seen in relation to one another. The last point is particularly impor- tant. We hoped to discover traffic pat- terns in reference use, and in addition more detailed information about users and the service provided for them-how the kinds of questions asked by various user categories differed, which categories of questions required the greatest searching time, where librarians looked for answers to inquiries, and so on. We also hoped that the study would contribute toward answering some of the persistent management problems confronting the Yale Library. Among these were: Is the level of staffing at the reference desks appropriate to the vol- ume of business? Would directories and signs aid users who have simple direc- tional inquiries? Should access to the li- brary by users not affiliated with Yale be restricted? Are the various reference collections useful, and are they properly located? Do users have the library skills they need to function in a research li- brary, or would library instruction be of benefit to them? We recognized that problems like these are complex, and that solutions to them depend on many factors. It seemed evident, however, that knowing about patterns of reference use would help in solving them. METHODS The approach that was taken to sat- isfy the design requirements was to use an interview forni which could be filled out during normal treatment of refer- ence inquiries and could double as a keypunching worksheet; to punch data on Hollerith cards, later transferring it to magnetic tape; and to use an existing computer program which would build bivariate frequency tables, permitting various items of data to be seen in rela- tion to one another. (The methodology described here borrows from techniques developed by the Research Department of the Yale University Library, the methods having been used for the anal- ysis of data collected during a study of catalog use conducted from 1967 to 1969. The authors are particularly in- debted to Peter Stangl, then of the Yale Medical Library, for his assistance in developing the methodology.) Collecting Data The worksheet is shown in Figure 1. Several months were spent developing and testing preliminary versions before this format was chosen. The only ques- tion asked of the user, other than those related to the reference inquiry, was affiliation. The remainder of the form was filled out by the librarian, usually at the conclusion of the inquiry. A form was completed for each person Reference Service I 11 who asked a question at the reference desk each time he or she asked, whether a form had previously been filed or not; a form was filled out for each inquiry, no matter how trivial. The form was designed to act as a worksheet for keypunching. For this reason, punched card columns are indi- cated for each field, and the specific coding to be punched in each field is given at the left-hand side of each data item. For example, a librarian recording an inquiry at 3:00p.m. would check the box under the Time of Day field for 2:00-4:00 p.m.; the keypuncher would punch cc05" in columns 2-3 of the card representing that inquiry. Most categories on the worksheet are self-explanatory, but a few words of clarification are necessary. Under Patron Affiliation, a division was made between persons who had official affiliation with the university (coded 01 through 07) and those who did not (coded 08 through 11). Cardholders are those per- sons who are not formally affiliated with the university but have borrowers' cards because they fall within special catego- ries qualifying for this privilege, or be- cause they have purchased cards. Under Type of Inquiry, a general en- try for card catalog inquiries (coded 03) was used for inquiries which did not fit any of the three more specific card catalog entries, or which involved several different kinds of activity at the catalog. Similarly, a general biblio- graphic entry (coded 07) was provided for inquiries which did not fit the more specific bibliographic categories. The c'IPL & Book ordering" entry (coded 11) was used both for interpre- tation of entries appearing on the IPL and for assistance in completing book request forms. The IPL (In-Process List) is one product of the Yale Univer- sity Library's computer-based Machine- Aided Technical Processing System ( MA TPS ) , whose purpose is to aid in the acquisition and processing of li- 12 I College & Research Libraries • January 1975 DAY OF WEEK (Column 1) 1[ ] Monday 2 [_1 Tuesday 3[ 1 Wednesday 4 [ 1 Thursday 5[_1 Friday 6 [=] Saturday 7 [==] Sunday TIME OF DAY (Columns 2-3) 01[ ] 8:30-10:00 02[==] 10:00-12:00 03[_] 12:00-1:00 04[ ] 1:00-2:00 05[=] 2:00-4:00 06[_1 4:00-5:00 07[_1 5:00-6:00 08[-] 6:00-7:00 - 09[ l 7:00-9:00 - 10[==1 9:00-10:00 MODE OF INQUIRY (Column 4) 1[=] Person 2 [=1 Telephone 3[=] Dept. Lib. DURATION (Column 5) 1[ 1 Negligible 2[-1 1-2 minutes 3[_1 3-5 minutes 4[=1 6-10 minutes 5[=] 11-60 minutes 6[_1 Over 60 minutes PATRON AFFILIATION (Columns 6-7) 01[-1 Yale Undergrad 02 [=] Yale Graduate 03[==1 Yale Faculty 04[=] Yale Staff 05[-] Yale Library Staff 06[ ] Immed. Family of Above 07[ ] Yale Altmmi 08[ ] Undergrad - Other Univ. 09[_1 Graduate - Other Univ. 10[=1 Faculty - Other Univ. 11[ ] Cardholder 12 [ 1 Other TYPE OF INQUIRY (Columns 8-9) 01[=1 1. General Information 02[ ] 2. Library Directions 03[ ] 3. Card Catalog 04[ ] A. Simple 05{-] B. Instructions 06[ ] C. Problem entries 07[ 1 4. Bibliographic 08[=] A. Citation 09[ 1 B. Instructions 10[ 1 C. Recommendations 11[ ] 5. IPL & Book ordering 12[ ] 6. Data 13[=1 7. Yale dissertations 14[-1 8. Referrals -15[ 1 9. Interlibrary Loans - 16[ ] 10. Stacks 17[=1 11. Library Instruction 18[ =1 12. Other Fig. 1 Sample Worksheet SEARCH LOCATIONS (Columns 10-11; 12-13; 14-15; 16-17) 01[ ] No search 02[-] Card Catalog 03[_] Index Collection 04[-] Catalog Reference Area 05[-] Main Reading Room 06[= ] Bibliography Room 07[ ] IPL 08[ ] Desk Reference Area 09 [ ] Technical Services 10[ ] Reference Office 11[-] Stacks 12[-] Other COMMENTS brary materials. The IPL is a main en- try list of materials which are on order, or which have been received and are at some point in the processing flow. In its present form, the IPL consists of a cartridge of 16mm computer output mi- crofilm containing some 85,000 entries, issued every two weeks; a microfilm reader; and a daily cumulative printout of orders generated since the last edi- tion of the microfilm. The "Data'' entry (coded 12) was used for inquiries as to specific facts, such as addresses, numerical data, statis- tics, biographical information, quota- tions, allusions, financial information, dates-all the inquiries that are often called c'ready reference." The entries under the Search Loca- tions category refer to areas on the ground floor of Sterling Library. Be- cause a large collection of books housed in the Reserve Book Room was being re- classed before being sent to a new in- tensive use library (called the Cross Campus Library) in the summer of 1970, an entry ~~Cross Campus Library" was added to the T.ype of Inquiry and Search Locations categories for Novem- ber 1970 and continued for the remain- der of the study. This entry appears in some of the tables, although it is not on the worksheet. In each category on the worksheet, li- brarians were instructed to make only one entry, with the exception of the Search Locations category, where they were instructed to make at least one, but not more than four, entries. It will be seen that in some categories (e.g., Dura- tion and Type of Inquiry) considerable judgment was required on the part of those completing the forms. Librarians were instructed to be as accurate as pos- sible in estimating time spent, and to exercise their best professional judg- ment in choosing inquiry type. Interview Method Librarians were instructed to conduct themselves at the reference desk in their usual manner as nearly as possible. No special policies or procedures were in- stituted for test purposes. The only deviation from normal practice was to ask the user~ s affiliation. Librarians were instructed to question the user closely at some point during the interview to determine his or her cate- gory as a user, and to inform the user if asked that the information was need- ed in connection with a study of library use which was being conducted. Only a few users wanted more information than that; most had a positive response to the idea of a survey. Librarians then proceeded with normal negotiation of the question, completing the remainder Reference Service I 13 of the form at the conclusion of the in- quiry. Completed forms were collected at the end of each day. Sampling Strategy Initially, three weeks in April 1970 were chosen as the period during which the study would be conducted, since this tended to be the busiest period for reference service. April also had been the period usually used to collect refer- ence statistics for annual reporting pur- poses. It seemed advisable, therefore, to use the same period for this survey in order to compare our findings with those of prior years. Two things became immediately apparent. First, there could be no comparison with prior years, since the data we collected was much more complex than any that had been gath- ered before, and it was collected much more carefully. Second, a three-week test in spring term might yield some use- ful information about reference use at that time of year, but it would also be desirable to know whether the patterns were different at other times. According- ly, four additional test weeks were chosen: one in November 1970, in order to observe patterns during the fall term; one in January 1971, during the reading period for exams; and two in April1971, in order to see whether there was any variation between the two spring terms. Since one of the aims of the study was to gather data about total reference traffic, it seemed desirable to select several entire weeks in which the total population could be recorded. No other method of selection was em- ployed. It must be acknowledged that the sam- ple obtained in this manner may not be as representative of the entire popula- tion as one might wish. We ·operated, however, in the environment of a work- ing library, where many other activities were being carried on at the time the study was conducted, and by the same staff. It was essential, therefore, to col- 14 I College & Research Librar·ies • January 1975 lect data at a time when it was possible to do so; tours for new students and lec- tures on library use take priority early in the fall, for example, to the effective exclusion of other special activities, making November the earliest date in the fall at which we could concentrate on this study. It should also be pointed out that there was remarkably little variation among the weeks chosen for the study, a circumstance that casts doubt on whether more careful sam- pling would have yielded substantially different results. Keypunching At the conclusion of each statistical period, worksheets were delivered to a keypunch operator. Verbal instructions were given to the operator; since card design was straightforward, and col- umns and punch codes were given on the worksheets, no difficulties were en- countered. Compilation of Tables Data on punched cards were analyzed by using an existing computer program available at the Yale Computer Center. This program compiles various kinds of statistical tables and is described in Yale Computer Center Memorandum No. 38, "Table Program" (Feb. 1968). Sample tables are shown in Figures 2 and 3. It will be seen that the program enables the computer to construct bivariate fre- quency tables comparing any two vari- ables in the data submitted to it. In the examples shown, Day of Week and Time of Day categories are compared. The table shows that between 2:00 and 4:00 on Wednesday afternoons, 241 users asked some kind of reference question; this was 4.7 percent of the in- quiries asked for the duration of the study, 26.6 percent of the inquiries asked on Wednesdays, and 18.1 per- cent of the inquiries asked between 2:00 and 4:00p.m. CosTs No additional personnel were added to the staff of the Reference Depart- ment for purposes of this study; there were thus no direct personnel costs. Di- rect costs were as follows: Printing of worksheets $50.00 Keypunching 70.00 Computer time, fiscal 1970 I 71 66.00 Computer time, £scal1971/72 56.00 Total $242.00 FINDINGS The data gathered during this study and the tables prepared by the computer were rather voluminous: 5,096 observa- tions were recorded, from which the computer prepared seventy-eight pages of tables. This by no means exhausted the possibilities for analysis of the data, but the tables which were prepared re- corded the data we believed would be most immediately useful. The findings below, and the associated figures, are based on the tables compiled by the computer, and are those which we con- sider to be of general interest outside the Yale environment. Variation Except as noted below, there was lit- tle variation in the data gathered · dur- ing the four periods of the study. Traffic was lighter during the week of January 1971 than during any other period of the study. This week fell dur- ing a reading period prior to examina- tions; reference use would appear to de- cline at such a time. Table 1 gives the number of inquiries for each reporting period. Use by Yale graduate students rose slightly during the winter months (No- vember 1970 and January 1971) but de- clined slightly during the spring. Use by persons not affiliated with Yale was high- er in the spring than in the winter. Use made of the reference service by out- siders was lower during the period of Reference Service I 15 REFERENCE USE STUDY --------------------=------------------------------------------------- -··. FREQUENCY TABLES _f!.Q.'i_'[~~!.~I!'=.~--I!~'!.~~-----------~Q..'=._'{.~~!.~I!.~t--lL'it _____________________ _ , ... _'!~'=.~ _____ Q.!_ ____ !_!. ____ ~!. ____ l!, ____ '!_!, ___ 2_!. ____ ~!. ____ l!, ____ ~!, ____ ~!. ___ ~p-~_ ·- . 0 0 0 0 0 0 0 1 1 0 0 0 2. ____ l------~----~~---~~~----~l----~~---t~~----1~----'!.t ____ '!.~---l~~----~~--~~~~ ____ l ______ Q. ____ ~~---ll~----~~----~~---tl~----~l----~1----l~----~~----~~-- -~~l 3 4 53 178 83 ~241 77 45 24 73 38 906 ----~------~----~~---lil ____ ~l----~~---~ll ____ l! ____ :!_~ ____ l! ____ ~'!.----~~---~l~ 5 6 51 157 77 Cl5 200 93 10 10 1 q 5 723 ---------------------------------------p~--~-~----------------------------- - 6 0 39 116 48 51 15! 6'7 0 0 0 0 473 • ___ 7_ ______ Q. _____ q_ _____ Q. _____ Q. _____ Q. _____ q_ _____ q_ _____ Q. _____ ~---~-1_~ --- _:g __ -.- ,_l '!.'!. -~l!'!~-----!.~ ___ l!.f! ___ ~~s ___ '!.l~---~q_t __ t~~l! ___ '!.!~ ___ tl!! ___ !.'!.~---~1_~ ____ 2_1_.'! -- - ·~oq6 CEll PERCENTAGES BY TOTAL 0 o.o o.o o.o o.o o.o o.o o.o o.o o.o o.o o.o o.o 1 o.1 1.3 4.o . 1.6 1.1 5.3 1.6 o.a o.q 2.s o.9 20.5 ------------------------------------------------------------------------------ 2 o.o 1.1 2.1 t.6 t.a 4.6 1.6 o.q o.6 1.1 o.a 17.4 ------------------------------------------------------------------------------ ' 3 o.t 1.0 3.5 1.a 1.6 4.7 1.5 o.q o.5 1.4 o.1 11.a ----~----~~!.---~~!.---~~~----!.~~ ___ !.~~---~~~---!.~~---q_~'!. ___ q_~~---!.~f!. ___ !.~~--!.f!~~ ----~----~~!. ___ t!~---~!!. ___ !.~~---~~~---~~~---!.~f!---~~~---~~~---~~~---~~!_ __ !.~~~ ____ 6 ______ o_.~o _____ o_.~a ______ 2~·3 _____ o __ .9 1.0 3.0 1.3 o.o o.o o.o o.o 9.3 1 o.o o.o o.o o.o o.o o.o o.o o.o o.o 2.2 0.6 2.8 ------------------------------------------------------------------------------ SUMS 0.4 8.6 9.8 26.1 3.6 2.8 10.0 4.2 100.0 Fig. 2 Sample Printout: Frequencies and Percentages by Total 16 I College & Research Libraries • January 1975 ---------.----------------------~~E~~fi<;_fi_t!~li-~ll!l!!. ____________________________ _ CELL PERCENTAG~S BY RON -~q~-~~~t~~~~--~~!~~-----------~q~-~~~l~~~~--l~~----------------------------- · .Y~~~-----~! ____ t! ____ ~! ____ l! ____ ~!---~!----~! ____ l! ____ ~!----~! ___ l~! ______ _ 0 o.o o.o o.o o.o o.o · o.o 5o.o 5o.o o.o o.o o.o 100.0 ____ t ____ ~~~---~~~--l~~l ___ r~~---~~l--~~~r ___ r~~---1~~---~~~--t~~~---~~~-t~~~~- ----~----q~~---~~~--t~~l---~~~--~~~~-~~~~---~~t---~~l---~~~---~~~---~~~-t~~~~ - 3 0.4 5.A 19.6 9.9 9.2 26.6 8.5 5.0 2.6 8.1 4.2 100.0 ----~----Q~I---~~~--t~~~---~~r __ t~~~--~~~~---~~~---~~! ___ 1~~--t~~!---~~t_t~~~~ - ____ ! ____ ~~~---r~t __ ~t~r __ tQ~r __ tl~t __ ~r~r __ t~~~---t~~---t~~---~~~---~~r_t~~~~ - 6 o.o ~.2 24.5 10.1 10.8 32.1 14.2 o.o o.o o.o o.o 100.0 ____ r ____ ~~~---~~~---~~~---~~~---~~~---~~~---~~~---~~~---~~~--tr~~--~~~~-t~~~~ - -~~~~----q~~---~~~--t~~~---~~~---~~~--~~~t---~~! ___ 1~~---~~~--t~~~---~~~-t~~~~ - CEll PERCENT~GES BY COLUMN -~Q~-!~~1~~~~--~~!~~-----------~~~-~~~l~~~~--ll~~----------------------------- · -~!~~-----Q! ____ t! ____ ~! ____ !! ____ ~! ____ ~! ____ ~! ____ l! ____ ~!----~! ___ 1~! ______ , 0 o.o o.o o.o o.o o.o o.o 0.2 0.5 o.c o.o o.o o.o ----~---lt~~--ll~~--l~~~--ll~~--l~~~--ll~~--l~~~--~l~l--~~~~--t~~~--~~~1--l~~~ . ----~---ll~~--t~~~--l~~l __ tr~~--t~~~--t~~t--l~~~---!~~---~~~---!~I---~~1--t~~~ . 6 o.o 1~.3 12.1 10.9 10.2 11.4 14.1 o.o o.o o.o o.o Fig. 3 Sample Printout: Percentages by Row and Column TABLE 1 NuMBER OF INQUIRIES, BY REPORTING PERIOD Average Number of Number of Number Inquiries Reporting Period Inquiries of Weeks per Week April 1970 2,252 3 751 November 1970 728 1 728 January 1971 566 1 566 April 1971 1,550 2 775 ~- Total 5,096 7 728 lowest use (January 1971) than during any other time, both in frequency ( 48 inquiries) and percentage ( 9.2 percent). Traffic The day of heaviest use was Monday, followed by Thursday, Wednesday, Tuesday, and Friday. Use declined on weekends: Saturday showed only 473 ob- servations, about half the Monday- Thursday average, and Sunday only 144 observations. This is partly a function of reduced hours on Saturdays and Sun- days, but the per-hour figure declined as well. (See Table 2.) Periods of heavy use occurred at 10:00 a.m.-12:00 noon, at 2:00-4:00 p.m., and at 7:00-9:00 p.m. Troughs oc- curred . at the dinner hour and after 9:00 p.m. More than one-fourth of all inquiries came during the peak after- noon period (2:00-4:00 p.m.); 17 per- cent came after 6:00 p.m. (See Table TABLE 2 NuMBER OF INQUIRIES, BY DAY OF WEEK Average Number of Number of Percent Inquiries Day of Week Inquiries of Total Rank per Hour Monday 1,045 20.5 1 11.1 Tuesday 887 17.4 4 9.4 Wednesday 906 17.8 3 9.6 Thursday 916 18.0 2 9.7 Friday 723 14.2 5 9.7 Saturday 473 9.3 6 7.9 Sunday 144 2.8 7 6.9 Not recorded 2 -- Total 5,096 100.0 Reference Service I 17 TABLE 3 NuMBER OF INQUIRIES, BY HouR OF DAY Number of Percent Hour of Day Inquiries of Total Rank 8:30-10:00 a.m. 10:00 a.m.-12:00 noon 12:00 noon-1:00 p.m. 1:00-2:00 p.m. 2:00-4:00 p.m. 4:00-5:00 p.m. 5:00-6:00 p.m. 6:00-7:00 p.m. 7:00-9:00 p.m. 9:00-10:00 p.m. Not recorded Total 318 962 439 501 1,328 476 183 145 511 214 19 5,096 6.2 18.9 8.6 9.8 26.1 9.3 3.6 2.8 10.0 4.2 0.2 100.0 7 2 6 4 1 5 9 10 3 8 3.) On Saturdays, periods of heavy use occurred at 10:00 a.m.-12:00 noon and at 2:00-4:00 p.m., as on weekdays; there was no staffing after 5:00 p.m. Sunday use was heavier than expected; refer- ence service was provided only from 7:00-10:00 p.m., but the number of in- quiries compared favorably with the same period for other days of the week. Most inquiries were delivered in per- son, about 84 percent; about 15.5 per- cent came by telephone. Telephone in- quiries declined after 5:00 p.m. and on weekends. Users As shown in Table 4, 77.5 percent of the users were affiliated with Yale; 14.3 percent were not. If unrecorded users are excluded, the percentages are: Yale, 84.7 percent; non-Yale, 15.3 percent. The relative importance of outsiders was greater on Saturdays, when they made up 26.8 percent of the user group, than TABLE 4 YALE VERSUS NON-YALE USERS Number of Percent Users Inquiries of Total Yale 3,962 77.5 Non-Yale 715 14.3 Not recorded 419 8.2 Total 5,096 100.0 18 I College & Research Libraries • January 1975 TABLE 5 YALE VERsus NoN-YALE UsERS, BY DAY OF WEEK 0 Yale Day of Number of Percent of Week Inquiries Daily Total Monday 825 86.5 Tuesday 722 87.9 Wednesday 710 85.9 Thursday 706 85.5 Friday 537 81.6 Saturday 331 73.2 Sunday 130 94.2 Total 3,962 84.7 0 Unrecorded users excluded. during the week. (See Table 5.) On Saturdays, however, the number of in- quiries by Yale users declined to less than half its weekday level, while the number of inquiries by outsiders stayed at about its weekday level ( 121 inquiries versus a weekday average of 117). The larger precentage is, therefore, a function of a smaller population. Use by outsid- ers declined abruptly on Sundays. Dur- · ing the working day ( 8:30 a.m.-5 :00 p.m.), periods of heavy use by outsiders; occurred at 10:00 ~.m.-12:00 noon and 2:00-4:00 p.m.; that is, at the same time Yale use was heaviest. (See Table 6.) Use by outsiders declined during the evening. Until 4:00p.m. outsiders made up about 17 percent of the user popula- tion, distributed fairly evenly through- Non-Yale Number of Percent of Inquiries Daily Total 129 13.5 99 12.1 117 14.1 120 14.5 121 18.4 121 26.8 8 5.8 715 15.3 Daily Total 954 821 827 826 658 452 138 4,677 out the day; from that point on, the per- centage of outside use declined steadily. Yale students accounted for more than half of all reference use; 35.5 per- cent of all users were Yale undergradu- ates, 24 percent were Yale graduate stu- dents. Detailed counts are given in Ta- ble 7. Use by Yale undergraduates rose sharply in the evenings. During the working day, Yale undergraduates made up 30.5 percent of the user population; for the 7:00-9:00 p.m. period, the per- centage rose to 61.8, and for 9:00-10:00 p.m., to 63.6. Use by Yale undergradu- ates rose dramatically on Sundays, to 72.2 percent of the user population, more than twice the normal level. Use by graduate students was steady throughout the day; they made up TABLE 6 YALE VERsus NoN-YALE UsERs, BY HoUR OF DAY 0 Yale Non-Yale Number of Percent of Number of Percent of Daily Hour of Day Inquiries Daily Total Inquiries Daily Total Total 8:30-10:00 a.m. 229 81.2 53 18.8 282 10:00 a.m.-12:00 noon 700 82.9 144 17.1 844 12:00 noon-1:00 p.m. 323 80.8 77 19.2 400 1:00-2:00 p.m. 377 82.9 78 17.1 455 2:00-4:00 p.m. 1,042 83.3 209 16.7 1,251 4:00-5:00 p.m. 374 85.6 63 14.4 437 5:00-6:00 p.m. 152 87.4 22 12.6 174 6:00-7:00 p.m. 113 87.6 16 12.4 129 7:0~9:00p.m. 450 91.8 40 8.2 490 9:00-10:00 p.m. 195 95.1 10 4.9 205 Not recorded 7 3 10 Total 3,962 84.7 715 15.3 4,677 0 Unrecorded users excluded. TABLE 7 NuMBER OF INQUIRIES, BY AFFILIATION OF UsER Affiliation of User Yale undergraduates Yale graduate students Yale faculty Yale staff Yale Library staff Immediate family of Yale users Yale alumni Undergraduates, other universities Graduate students, other universities Faculty, other universities Cardholders Other Not recorded 0 Total Number of Percent Inquiries of Total Rank 1,810 35.5 1 1,223 24.0 2 360 7.1 4 151 3.0 7 321 6.3 5 59 1.2 12 38 0.7 13 138 2.7 8 116 2.3 9 71 1.4 11 92 1.8 10 298 5.8 6 419 8.2 3 5,096 100.0 0 Affiliation of the user was not asked when the in- quiry came by telephone. about one-fourth of the users for any given time period, and the figure never rose above 30 percent or dropped below 20 percent. Two general conclusions: ( 1) Officials of the Yale Library have sometimes as- serted that outside users pour into the library on evenings and weekends~ We found that, so far as reference service is concerned, outside use declines in the evenings and on Sundays, and that peri- ods of heavy or light use by outsiders follow the pattern set by Yale users; Figure 4 illustrates this. ( 2) The group utilizing the reference service in the evenings and on Sundays was made up predominantly of persons who reside on or near the campus; that is, Yale un- dergraduates. Duration Only nine inqutnes ( 0.2 percent of the total) required more than sixty min- utes of searching time. (See Table 8.) Of all inquiries, 9.8 percent required more than five minutes of searching time. In other studies, the percentage of inquiries requiring more than five min- Reference Service I 19 TABLE 8 NuMBER OF INQUIRIES, BY DURATION OF SEARCH Number of Percent Duration of Search Inquiries of Total Negligible 1,289 25.3 1-2 minutes 1,942 38.1 3-5 minutes 1,334 26.2 6-10 minutes 352 6..9 11-60 minutes 137 2.7 Over 60 minutes 9 0.2 Not recorded 34 0.7 Total 5,096 100.0 Rank 3 1 2 4 5 7 6 utes of searching time was on the order of 2 to 3 percent.8 A number of fac- tors may account for this difference: in- accuracy in recording the time spent; the size of Sterling Library; the fact that the cited study was conducted in undergraduate libraries while ours was conducted in a research library; etc. Of all inquiries, 25.3 percent could be answered at once ("Negligible" dura- tion). Questions posed by undergraduates re- quired relatively little searching time; those by graduate students, slightly more; and those by faculty, more searching time than questions from any other group. Inquiries Information-direction questions ("Gen- eral information" and "Library direc- tions" categories) made up 30.6 percent of inquiries. (See Table 9.) One inquiry in four related to the card catalog. Bibliographic inquiries made up 16.4 percent of the total. In other studies, inquiries for specific facts made up fairly large proportions of the questions asked. 9 We found that inquiries for data were relatively rare at Sterling Library, only 6.3 percent of the total. Once again, the difference may be in part a function of the difference in population between our study and others. Users who phoned in their in- N u M B E R 0 F I N Q u I R I E s 20 I College & Research Libraries • January 1975 1080 960 840 720 600 480 360 240 120 ,. 1\ I \ I \ I' I \ I ' I \ I ', I \ I , _____ 1 \ I \ II \, -Yale ----Non-Yale '........ ----- ....... , ____ .,. ............ _ 10:00- 12:00 1:00-- 2:00 TIME OF DAY Fig. 4 4:00- 5:00 6:00-- 7:00 Yale Versus Non-Yale Users, by Time of Day 9:00-- 10:00 TABLE 9 NuMBER OF INQUIRIES, BY TYPE OF INQUmY Number of Percent Type of Inquiry Inquiries of Total Rank General information 571 11.2 3 Library directions 987 19.4 1 Card catalog (General) 133 2.6 14 Simple 817 16.0 2 Instructions 190 3.7 11 Problem entries 200 3.9 9 Bibliographic (General) 140 2.7 13 Citation 260 5.1 6 Instructions 62 1.2 16 Recommendations 377 7.4 4 IPL & book ordering 170 3.3 12 Data 322 6.3 5 Yale dissertations 115 2.3 15 Referrals 220 4.3 7 Interlibrary loans 37 0.7 18 Stacks 193 3.8 10 Library instructions 37 0.7 18 Cross Campus Library 0 54 1.1 17 Other 201 3.9 8 Not recorded 11 0.2 20 Total 5,096 100.0 ° Category used during November 1970 and April 1971. quiries made a higher percentage of data inquiries than those who came in person ( 17.5 percent of telephone in- quiries were classed as data inquiries, only 5 percent of inquiries in person). As one might expect, information-di- rection questions could be answered quickly, while more substantive card catalog or bibliographic inquiries re- quired more time. Thus 90.2 percent of the "General information" and 89.7 percent of the "Library directions" questions were answered in two minutes or less. By way of contrast, only 32.1 percent of the general bibliographic in- quiries could be answered in less than two minutes. It is interesting to note that of questions in the information- direction categories, 191 ( 10.7 percent) required three minutes or longer to answer; of these, 19 required six to ten minutes, and 3 required eleven to sixty minutes. These inquiries may have be- gun as simple directional inquiries and later developed during interrogation into more complex problems, but have Reference Service I 21 been entered, through oversight or oth- erwise, as directional inquiries. Never- theless, it cannot be assumed that be- cause an inquiry appears to be a simple information-direction question it always permits a rapid answer, or that the training of a professional librarian will be wasted in pursuing the answer. We found it interesting to compare the inquiries posed by Yale students and faculty (see Table 10). More than half the questions about stack problems and more than half the requests for recom- mendation of bibliographic resources were asked by Yale undergraduates, as were two-thirds of the Cross Campus Li- brary questions. Categories of inquiries that Yale undergraduates tended to ask were: general information, library di- rections, simple catalog lookups, cita- tion problems, bibliographic recommen- dations, data, and stack problems. This may be taken as a more or less classical undergraduate use pattern: undergradu- ates are confused by the library and re- quire considerable help in finding their way around in it; they are unfamiliar with card catalogs and require assistance with the simplest lookups; they have difficulty finding materials in the stacks; they know little about bibliographic re- sources. In other words, they are unfa- miliar with the rudimentary mechanics of library use. Questions asked by Yale graduate stu- dents followed the undergraduate pat- tern closely, except that graduate stu- dents asked higher percentages of the inquiries relating to the IPL (In-Process List), to Yale dissertations, and to inter- library loans. Categories of inquiries that graduate students tended to ask were: general information, library di- rections, simple catalog lookups, prob- lem entries at the catalog, citation prob- lems, bibliographic recommendations, and IPL and book ordering. Some of the differences between graduate stu- dents and undergraduates are not sur- 22 I College & Research Libraries • January 1975 TABLE 10 INQUIRIES BY YALE STUDENTS AND FACULTY: FREQUENCY, Row PERCENTAGE, AND CoLUMN PERCENTAGE, BY TYPE OF INQUIRY Type of Inquiry General information Library directions Card catalog (General) Simple Instructions Problem entries Bibliographic (General) Citation Instructions Recommendations IPL & book ordering Data Yale dissertations Referrals Interlibrary loans Stacks Library instructions Cross Campus Library Other Not recorded Total Yale Undergraduates Number Percent Percent of by by Inquiries Row Column 201 35.2 11.1 433 43.9 23.9 43 32.3 2.4 208 25.5 11.5 75 39.5 4.1 55 27.5 3.0 43 30.7 2.4 91 35.0 5.0 32 51.6 1.8 175 46.4 9.7 52 30.6 2.9 95 29.5 5.2 15 13.0 0.8 61 27.7 3.4 9 24.3 0.5 105 54.4 5.8 10 27.0 0.6 36 66.7 2.0 69 34.3 3.8 2 18.2 0.1 1,810 35.5 100.0 pnsmg: graduate students are presum- ably more aware of the literature of their field, and hence are more likely to inquire about books being processed and to consult the IPL; they are en- gaged in doctoral research, so they are interested in the dissertation literature. The differences between the use pat- terns shown by graduate students and by undergraduates, however, are not nearly so striking as the similarities. One might expect that the greater sophisti- cation and constant exposure to libraries and the scholarly record which are pre- sumed to characterize graduate students might change the way in which they use the library. We found to the contrary that graduate students, like undergradu- ates, ask many information-direction questions and require elementary help with the catalog and in finding biblio- graphic resources. The significance of this finding is difficult to assess in the absence of detailed knowledge about the motivations and work patterns of Yale Graduate Students Yale Faculty Number Percent Percent Number Percent Percent of by by of by by Inquiries Row Column Inquiries Row Column 109 19.1 8.9 27 4.7 7.5 236 23.9 19.3 70 7.1 19.4 22 16.5 1.8 11 8.3 3.1 128 15.7 10.5 63 7.7 17.5 50 26.3 4.1 13 6.8 3.6 62 31.0 5.1 24 12.0 6.7 48 34.3 3.9 6 4.3 1.7 92 35.4 7.5 10 3.8 2.8 19 30.6 1.6 4 6.5 1.1 110 29.2 9.0 20 5.3 5.6 69 40.6 5.6 31 18.2 8.6 50 15.5 4.1 23 7.1 6.4 49 42.6 4.0 7 6.1 1.9 35 15.9 2.9 12 5.5 3.3 21 56.8 1.7 1 2.7 0.3 54 28.0 4.4 15 7.8 4.2 10 27.0 0 .8 3 8.1 0.8 12 22.2 1.0 5 9.3 1.4 45 22.4 3.7 15 7.5 4.2 3 27.3 0.2 0 1,224 24.0 100.0 360 7.1 100.0 both groups. It seems proper to con- clude, however, that the obstacles which cause difficulty for undergraduates (size and complexity of the building and col- lections, reflected in the size, number, and complexity of the catalogs and oth- er bibliographic aids) still cause prob- lems, and that graduate students have not learned much better than their un- dergraduate counterparts the rudimen- tary skills that are needed in using a re- search library. Yale faculty asked large proportions of problem catalog entry questions and IPL inquiries. Categories of inquiries that Yale faculty tended to ask were: general information, library directions, simple catalog lookups, problem entries, bibliographic recommendations, IPL and book ordering, and data. This pat- tern departs from the graduate-under- graduate pattern in the higher incidence of problem entries at the catalog and heavier use of the IPL and book order- ing procedures. TABLE 11 NuMBER oF SEARcHEs, BY SEARCH LocATION Number of Percent Search Location Searches of Total Rank No search 1,992 33.3 1 Card catalog 1,564 26.1 2 Index collection 346 5.8 6 Catalog reference area 435 7.3 4 Main reading room 455 7.6 3 Bibliography room 166 2.8 8 IPL 161 2.8 9 Reference desk area 405 6.8 5 Technical services 109 1.8 10 Stacks 73 1.2 11 Reference office 72 1.2 12 Reserve book room 2 0.0 14 Other 185 3.1 7 Not recorded 20 0.3 13 Total 5,985 100.0 Search Locations Tables of search locations were calcu- lated by merging the data provided by the computer, which tabulated four sets of location tables, one for each loca- tion marked on the survey worksheets. (See Table 11.) For each inquiry, an average of 1.17 search locations was used. One in three inquiries required no search. One inquiry in four required use of the card catalog. The index collection, catalog refer- ence area, main reading room, and the Reference Service I 23 collection of materials at the reference desk each accounted for about 7 per- cent of the search locations. Bibliogra- phy room and I·PL each accounted for about 3 percent. "General information" and "Library directions" queries were generally an- swered either after no search or from materials kept at the reference desk. Sixteen percent of card catalog in- quiries required the librarian to go to some resource other than the card cata- log, most frequently to the catalog ref- erence area or to technical services. Bibliographic inquiries required con- sultation of a wide range of resources: card catalog, index collection, catalog reference area, main reading room, and bibliography room ·were all heavily con- sulted. If the variety of resources used is a valid gauge, bibliographic queries would appear to be the most complex kind of reference problem. Other search locations followed di- rectly from the nature of the inquiry. Data inquiries required use of the ready reference collection in the main reading room or the reference desk area; Yale dissertation inquiries required use of the catalog reference area, where track- ing tools for Yale dissertations are kept, or the reference office, where card files and correspondence relating to disserta- tions are kept; etc. TABLE 12 SEARCH UNITS Monday Tuesday Wednesday Thursday Friday Saturday Sunday 8:30-10:00 a.m. 11.6 20.4 13.9 30.9 20.0 7.2 10:00 a.m.-12:00 noon 26.4 19.2 31.8 25.3 18.2 15.3 12:00 noon-1:00 p .m. 17.6 30.0 17.0 15.4 14.3 17.6 1:00-2:00 p.m. 22.7 19.7 22.3 32.9 13.8 18.8 2:00-4:00 p.m. 15.1 19.3 19.0 17.0 19.5 19.3 4:00-5:00 p.m. 20.5 14.6 21.9 16.4 18.4 10.5 5:00-6:00 p.m. 15.7 12.1 22.2 25.0 8.0 6 D0-7:00p.m. 16.2 18.3 5.8 17.4 6.7 7 00-9:00 p.m. 21.4 22.1 18.7 17.6 7.5 20.9 9 00-10:00 p.m. 21.5 22.2 13.6 20.9 4.3 13.6 Average 18.9 19.9 20.7 21.4 16.8 14.9 18.5 Overall hourly average: 19.1 24 I College & Research Libraries • January 1975 Search Units How much of their assigned desk time do reference librarians actually spend in reference work? In an attempt to answer this question a search units table was compiled (Table 12). The values for each cell were com- puted as follows. The number of in- quiries for each time period was tabu- lated. Weights were assigned to each query roughly corresponding to the mean time spent in searching for an answer, as follows: Negligible 1-2 minutes 3-5 minutes 6-10 minutes 11-60 minutes Over 60 minutes 0.5 1.5 4.0 8.0 35.0 60.0 The number of inquiries for each cell was multiplied by the appropriate weight then divided by the number of hours of assigned reference service to obtain the average time expended in searching during each time period. The search unit figures obtained in this way provided a rough approxima- tion of the amount of time spent in ac- tually negotiating reference inquiries. It should be remembered that searching time is derived from librarians' esti- mates of ·the time they spent looking for answers, and that the figures given in the cells of Table 12 do not repre- sent real time, but are artificial num- bers arrived at by manipulation of the weighting factors, which are themselves arbitrary. The temptation to think of the cell values as time spent in nego- tiating inquiries is all but irresistible, however. The following tentative con- clusions are offered: 1. The amount of time expended for each hour of assigned desk duty varied from 4.3 (Friday, 9:00-- 10:00 p.m.) to 32.9 (Thursday, 1:00-2:00 p.m.). In only three cells did the search unit figure ex- ceed 30. 2. The average search unit figure for each hour of assigned desk duty was 19.1. It seems reasonable to conclude that reference librarians spent about twenty minutes of each hour of desk time answering inquiries. 3. Search unit figures were very low for Friday evenings and Saturday mornings. Evening figures (with the exception of Fridays) com- pared favorably with those during the day, and Sunday figures com- pared favorably with weekday eve- ning figures. GENERAL CONCLUSIONS Staffing Because of the low number of in- quiries and the low search unit figures for Friday evenings and Saturday morn- ings, we concluded that it was not neces- sary to provide reference service after 5:00 p.m. on Friday or before 10:00 a.m. on .Saturday. As a direct result of this survey, staffing of the reference desks was stopped on Friday evenings in the fall of 1970. The principal group served is the Yale student body, undergraduates and graduate students. Use by this group, particularly by undergraduates, in- creases during the evening. Evening staffing of the reference desks by pro- fessional librarians should be contin- ued, except for Fridays. Both in terms of total inquiries and search unit figures, Sunday evening use compared favorably with weekday eve- nings. When one considers that the user group on Sunday evenings consisted mostly of Yale students, particularly un- dergraduates, it is clearly in order to continue Sunday evening staffing. Directory Information The use of directories and signs to provide basic information about the li- brary should assist in handling the high level of information-direction ques- tions. A building directory, an improved system of directional signs, and manuals for Sterling and other libraries have been generated, in part as a result of this study and using some of the data it provided. (During the January 1971 portion of the study, librarians were asked to record the exact nature of all information-direction questions. These inquiries were used in making direc- tories and floor plans for Sterling Li- brary.) Catalog Problems The card catalog is the single most important reference aid. Anyone who has used a catalog as large as the one in Sterling Lil;:>rary (about eight million cards) will appreciate that it is an un- wieldy and often baffling thing to use. If one considers the number of libraries the catalog is meant to serve, the size of the collections it is meant to describe, and the range of functions it is expect- ed to fill, one cannot fail to be im- pressed that the catalog performs as well as it does. We believe, however, that the level of problems we encoun- tered in its use is distressingly high ( 3.9 percent of all inquiries; 14.9 percent of catalog inquiries). The need for a thorough study of the problems readers encounter at the catalog is indicated. In order to assist readers in using the catalog, the Reference Department, with advice from the Catalog Depart- ment, has prepared two manuals, one on general card catalog use, the other a guide for locating serial titles. The ex- perience gained in this survey was use- ful in the preparation of these man- uals. Bibliographic Inquiries Taken together, inquiries which re- quired the consultation of bibliograph- ic sources (i.e., the card catalog, bibli- ographies and indexes, the IPL, disser- tation tracking tools, interlibrary loan Reference Service I 25 l verification, library instruction) made up 49.8 percent of all inquiries. Re- quests for data made up 6.3 percent, and requests for stack assistance another 4.8 percent. All these inquiries call for a measure of bibliographic or technical expertise, or both. It seems to us axio- matic that the presence of a trained pro- fessional librarian is essential to handle these inquiries. There remains a residue of some 40 percent, most of which were informa- tion-direction inquiries. It seems likely that paraprofessional assistants could handle these inquiries, and could direct other, presumably more difficult, ques- tions to reference librarians. The de- cision to use paraprofessional assistants at the reference interview points is not one that may be taken lightly, however. It has been the practice at the Yale Li- brary to staff the reference desks only with professional librarians, on the the- ory that only professional training pro- vides the background and the commit- ment that will enable the person behind the desk to interpret successfully the range of reference contacts he or she is likely to receive. Our study was incon- clusive on this point. Certainly there was a great number of information-di- rection questions; but as we noted above, some of them proved on interro- gation of the user to be more compli- cated than they at first appeared. We are not willing to change the present prac- tice of staffing reference desks with pro- fessional librarians because of our find- ings in this study, but we believe that the concept of a separate information desk staffed by paraprofessionals has merit and should be studied more in- tensively than this study has permitted. Instruction The level of library skills possessed by users, especially undergraduates, is not high. Formal library instruction would be beneficial to students at all 26 I College & Research Libraries • January 1975 levels. A program of instruction in bib- liographic resources has been begun at the Yale Library. Outside Use The level of use by persons not affili- ated with Yale ( 15.3 percent of all users) approximates that obtained in a recent study of catalog use conducted in Sterling Library.l0 It was not the in- tent of our study to treat outside users differently than they are normally treat- ed, or to provide a basis for excluding them or screening their use of the li- brary. Since the questions asked by out- siders tended to be uninvolved and to admit of ready answers, their use of the reference service does not interfere with our ability to serve the Yale user community. It may still prove desirable to screen outsiders on other grounds. Methods The methods used in this study were inexpensive, easy to administer, and in- terfered minimally with normal refer- ence work. The computer program used is a type that is widely available. The methods could therefore be adapted for use in other institutions. Utilization In addition to providing information for the management decisions discussed above, this study has been useful in ways we did not foresee. When it be- came necessary because of budget con- straints to reduce library hours in the fall of 1971, we were able to provide advice as to when hours might be cur- tailed with the least effect on our abil- ity to serve readers. Later in the same year, when students challenged the re- duction in hours of service and suggest- ed that library funds be saved by elim- inating evening reference service, the university librarian was able to use our study to demonstrate that evening use of the reference service was significant, and that the user community during the evening hours was made up predom- inantly of Yale students; evening refer- ence service was preserved. REFERENCES 1. Rutherford D. Rogers, "Measurement and Evaluation," Library Trends 3:177-87 ( 1954-55). 2. Samuel Rothstein, "The Measurement and Evaluation of Reference Service," Library Trends 12:456-72 (1963--64). 3. American Library Association, Section of Library Organization and Management, Statistics Coordinating Committee, Library Statistics: A Handbook of Concepts, Defi- nitions and Terminology (Chicago: Ameri- can Library Assn., 1966), p.24. 4. Edith Guerrier, "The Measurement of Ref- erence Service,'' Library Journal 61:529-31 (July 1936). 5. Herbert Goldhor, "Reference Service Anal- ysis," Illinois Libraries 42:319-22 ( 1960). 6. The Nelson Associates study of the New York Public Library, for example (User Survey of the New York Public Library Research Libraries [New York: Nelson As- sociates, Inc., 1969]), was intent on learn- ing such things as the age, sex, and occupa- tion of users. 7. Dorothy E. Cole, "Some Characteristics · of Reference Work," College & Research Li- braries 7:45-51 (Jan. 1946). 8. See, for example, Billy R. Wilkinson, Ref- erence Services for Undergraduate Students: Four Case Studies (Metuchen, N.J.: Scare- crow, 1972), p.96 passim. 9. Several studies are summarized by Louis Shores in his Basic Reference Sources: An Introduction to Materials and Methods (Chicago: American Library Assn., 1954), p.6-7. 10. Ben-Ami Lipetz, User Requirements in Identifying Desired Works in a Large Library (New Haven, Conn.: Yale Uni- versity Library, 1970 ), p.7. (Also available as ERIC document no. ED 042479.)