College and Research Libraries ORLANDO BEHLING and KERMIT CUDD A Library Looks at Itself In attempting to evaluate its service The Ohio State University li- braries in M01J 1966 designed a questionnaire to determine: (1) the characteristics of library users; (2) the ways in which users avail them- selves of the facilities; (3) the users opinions of the library and its services. The results indicated that: (1) 55 per cent were using the library only for study purposes; (2) this group was more dissatisfied than those seeking service or information; (3) history majors and faculty were the heaviest users followed by students in education, business, political science, and English; (4) the questionnaire is effective for measurement of library-campus relations. LIBRARIES EXIST for two fundamental reasons. As repositories of man's re- corded knowledge they seek to build representative collections of significant materials and preserve them for future generations. The Folger, Huntington, and Newberry libraries are obvious ex- amples of libraries where these roles of collection building and preservation are dominant. To be sure, libraries of this cype are in the minority. The majority of libraries have information dissemina- tion as their predominant function. In this role the libraries' holdings are or- ganized and controlled for maximum availability. For the majority of libraries, effectiveness is not determined entirely by size of the collection but rather by the success with which they are able to provide the user with the information he seeks. The library ful£lls this function best by pursuing a policy of constant sell- evaluation in order to keep alert to the changing needs of its users. The usual methods of evaluating performance are by internal criteria. Criteria frequently Dr. Behling is Assistant Professor, and Mr. Cudd is a doctoral candidate, in the Department of Business Organization of Ohio State University. 416/ considered include size and growth of the collection, circulation and reference statistics, new services added, and num- ber of library users served. Appraisal may also be concerned with the speed at which books are procured, processed, and made available to the public. These criteria are satisfactory for comparing library with library and present with past performance, if the library is con- sidered only as a repository. How suc- cessful are they in determining whether the library is providing the information its public needs and wants? In this re- spect, these traditional methods of ap- praisal are perhaps weak. At best, ap- praisal by internal criteria measures only indirectly a library's success as an infor- mation disseminating unit. How then is a library to determine the degree of success with which it serves its public? The ultimate authority, the li- brary user, is the most logical source of an answer. Libraries are the constant recipients of compliments, suggestions, and complaints. These unsolicited com- ments have formed the basis for passive appraisals to which libraries respond haphazardly. There are, however, many dangers in attaching credence to such random observations. In the first place, a library has no way of knowing whether or not the comments received are repre- sentative of all patrons. It may be that the library has heard from a small, but vocal, minority. Also to be considered is the propensity for people to verbal- ize complaints more frequently than compliments. It is possible that some areas of the library could be over looked completely because no one happens to comment, while other areas could be so ~ susceptible that they would receive con- tinual and, perhaps, unwarranted at- tention. Therefore, there is probability that this type of appraisal is not repre- sentative, is negatively biased, and is haphazard in areas of coverage. It would seem more logical to seek users' opinions and measure their attitudes in an active and systematic manner. Ohio State University libraries has at- tempted to take such an active approach to the measurement of user attitudes. With a system that includes a main li- brary and twenty-two departmental li- braries, The Ohio State University li- braries have a book collection of over 1,700,000 volumes, and serve a student body of 31,604 and 2,857 faculty. The effectiveness of its service to the campus could not be determined by the sporadic feedback that it received from students and faculty. It was decided, therefore, that it was necessary to develop a meth- od of actively determining library users' opinions. The technique employed to secure meaningful data was an opinion survey. How THE STUDY WAS PERFoRMED The study performed by The Ohio . State University libraries was designed to evaluate main library performance from the point of view of its users. Spe- cifically it was designed to gather infor- mation about: ( 1) the nature of indi- viduals who use the library; ( 2) the ways in which they make use of its fa- cilities; ( 3) their opinions about the li- brary and its services. A Library Looks at Itself I 417 A printed questionnaire was designed to be handed out to users as they entered the library with a request that they complete and return it at the guard station before leaving the building. Such an approach, using a questionnaire con- taining written instructions for its com- pletion, permitted reaching a large pro- portion of those using the library at a minimum cost. The questionnaire was designed for ease and economy of tabulation. Closed- end questions, requiring only that the li- brary user check one of a predetermined series of responses, were used for gath- ering most of the information. Such a format has the advantage of permitting machine tabulation of the data which again minimizes manpower costs. This type of question also has the advantage of requiring a minimum amount of the respondent's time, which probably in- creases the over-all return of usable questionnaires. An open-ended question which permits the respondent to write a full paragraph if he so wishes was used as the last item on the questionnaire. This was used to probe an area where there was considerable doubt as to the nature of the responses which might be expected. It was hoped that such a for- mat would minimize chances of distort- ing or concealing important aspects of the information requested. Provision was made, however, for numeric classification of the responses to this question so that machine tabulation techniques might be applied to them. The £rst group of questions was aimed at gathering information about the na- ture of individuals using the library . These questions, all of which were closed-end in format, were designed to determine the respondent's classification (freshman, sophomore, junior, senior, graduate student, faculty member, or other); his major £eld of study; and the frequency with . which he made use of library facilities. Information as to time of day (morning, afternoon or evening) 418 I College & Research Libraries • November 1967 was also obtained through the use of color-coded questionnaires. The informa- tion obtained from these questions was put to a dual use. First, it provided a basis for classifying opinion responses so that certain ideas about which groups would show the most favorable reactions to library services could be tested. Sec- ond, it was valuable in itself, since it pro- vided the first organized information about library users. The second set of questions concerned the use made of library facilities by the respondents. These questions, which were also closed-end in format, deter- mined whether the respondent visited the library in search of some specific information or service or whether he came for general study purposes. They also indicated, for those individuals seek- ing information or service, where they sought and where they found the infor- mation or service. The final group of questions dealt with the opinions of the respondents about the library and its services. Two closed- end questions permitted respondents who were seeking information or service to rate on a five-point scale ranging from very satisfactory to very unsatisfactory the ease of location of the information or services sought and the helpfulness and courtesy of staff members encountered. A third question, identical in format to the two preceding ones, permitted all respondents, regardless of whether they came to study or in search of informa- tion or services, to rate the over-all .. satis- factoriness" of their visit to the library. An additional question, for those indi- viduals seeking service or information, asked for the amount of time spent wait- ing. The final question, which was open- ended in format, permitted respondents to express in their own words what made their visit satisfactory or unsatisfactory. The questionnaires were passed out to every fifth indjvidual entering the library on a Thursday near the middle of Spring quarter 1966. This rate was determined by the ability of a single individual to hand out and explain briefly the purpose of the questionnaire. No attempt was made to restrict the number of question- naires to that which would provide only an .. adequate sample size." Pre-tests had I" revealed marked variations in the rate of return and thus, since there were mini- mal variations in the cost involved, it was considered best to obtain the maxi- mum number of responses. No attempt was made to stratify the sample, since no data existed on which to base the classifications. A total of 1,230 questionnaires were • handed out, of which 687 or 55 per cent were returned in usable form. This was considered an acceptable response, though somewhat lower than that ob- tained in the pre-tests of the question- naire. The data obtained was tabulated on an IBM 1620 computer. The program permitted the classification of data along the different dimensions discussed iri re- gard to the .. nature of users" questions . It provided frequency distributions as well as means and other parameters. Preliminary examination of the results revealed two facts which led to a change in the original plans for analysis. First, high correlatio11s were found among the answers to the three questions dealing with ease of obtaining service, courtesy and helpfulness of staff, and over-all sat- isfactoriness of the visit to the library. This led to the conclusion that the data from the question on ease of service was redundant, and it was therefore not in- cluded in the final report. It was. also found that, in contrast to pre-test respond- ents, a relatively small percentage of individuals, filled in the figures on wait- ing time. Therefore these figures were also omitted from the final report. The analysis of the data and the writ- ing of the final report were performed in the light of several cautions. First it was recognized that the sample taken was, in all probability, somewhat biased. Practical considerations made it impos- sible to sample from all days of the week and from all weeks in the quarter. Fur- ther, follow-up oral questioning of the non-respondents in one of the pre-tests revealed that there was a higher propor- tion of faculty members among the non- respondents than among the respondents and that the non-respondents · were slightly less likely to have a favorable over-all opinion of the library than were the respondents. Taken together, these things led to a decision to make no sta- tistical test of the differences found. It was felt that the presentation of such tests would lend a spurious sense of cer- tainty to the data. The analysis presented in the final report was, therefore, lim- ited to the presentation of the frequency distributions and associated means. WHO uSES THE LIBRARY? In interpreting the information pre- sented in this and in following sections, the reader should keep one important point in mind. These data concern one library on one university campus. Though there is some evidence available indicat- ing that similar patterns may occur in other libraries, 1 there can be no guaran- tee that the results obtained in the Ohio State University Study can be general- ized to all libraries on all campuses. When library users were grouped ac- cording to university classification, a pro- gressive decline in the absolute number of respondents occurred as one moved up the scale from freshman through soph- omore, junior, senior, and graduate stu- dent to faculty member. For the most part, this can be seen as a reflection of the decreasing total numbers of individ- uals in the classifications, since the ratio of the number of respondents to the number of individuals in a class re- 1 Results similar in several respects to those obtained in this study were found at the University of Dela- ware-see Gorham Lane, " Assessing the Undergradu- ates' Use of the University Library," CRL, XXVII (July 1966) , p. 277-81; and at Western Michigan University (personal correspondence to author from Katharine M . Stokes, director of libraries, Western Michigan University). A Library Looks at Itself I 419 mained relatively constant. To a certain extent, however, it also represented a change in the pattern of use of library facilities. While no consistent pattern was found among the variations in num- bers of respondents using the library as a source of information, materials, or services, a progressive decrease in the number of individuals using it for study purposes was uncovered. The study indicated that there was a core, particularly among those respond- ents using the library for study pur- poses, of heavy users who made demands on library space and facilities far out of proportion to their numbers. Two-thirds of those completing questionnaires had previously visited the library four or more times during the preceding two · weeks. Only 6 per cent of the respond- ents indicated, in contrast, that this was their only visit to the library during the two-week period. It is particularly interesting to note that though the proportion of those using the library for study purposes was fairly consistent in the infrequent visitor cat- egories, a marked shift occurred when the group which had made four or more visits was considered. In this group a far larger proportion of the users came for study purposes. The patterns of utilization of library facilities by respondents enrolled or em- ployed in various departments of the university are some of the most difficult to explain. Though much of the varia- tion · can be attributed to differences in relative enrollment, this does not ac- count for all of them. Certain large de- partments were represented by a rela- tively small number of respondents, while some small departments appeared in numbers out of proportion to their en- rollments. Attempts were made to ex- plain these differences in terms of the availability of departmental libraries, but, though this is doubtless a factor in the differences, it could not account for all of them. Unless these are purely 420 I College & Research Libraries • November 1967 chance variations, as is always possible, further . research will be required to ex- plain them. How Do THEY UsE LIBRARY FACILITIEs? One of the more surprising results of the study concerned the purposes of the visits made by respondents. Though there were marked variations from class to class and during different portions of the day, almost 55 per cent of those com- pleting questionnaires were in the library for general study purposes, rather than in search of specific information, mate- rials, or services. This represents a rad- ical departure from the information dis- seminating role of the university library and indicates a need for a reappraisal of the planning of libraries and of study facilities. The provision of study accom- modations in other facilities, such as dor- mitories and student unions, could re- lieve much pressure for space at a cost far below that of constructing and oper- ating additional library facilities. Heaviest utilization of library facili- ties came during the afternoon, followed by morning and evening, respectively. Average hourly use remained relatively constant during the day and dropped off during the evening. Marked shifts oc- curred among these periods in the pat- tern of utilization of library facilities. During the afternoon hours there was a disproportionately large increase in the demand for information, materials, and services. Variations in the patterns of utiliza- tion by individuals enrolled in the vari- ous departments of the university were also discovered. The results indicated that the history department majors and faculty were the heaviest users of the main library. The next four departments in order of usage were education, busi- ness organization, political science, and English. Again, however, no ready ex- planation was available for these varia- tions and thus the understanding of this aspect of the results will have to await further research. WHAT Do THEY TmNK OF THE LIBRARY? The interpretation of opinion survey data almost always presents problems. Rarely do absolute guidelines exist against which the obtained results may be judged in terms of "goodness" or "badness." Rather, the data have mean- ing only relative to those obtained for other groups or under other conditions. The Ohio State University Study was no exception to this rule and thus the dis- cussion of this data was limited to com- parisons of various aspects of the library operation and of different user groups. Nonetheless, it was heartening to the library staff, who were used to the com- plaint-oriented passive methods of eval- uating library-user attitudes, to see the favorable over-all evaluation of the li- brary and its services. The first opinion question dealt with the respondents' evaluation of the satis- factoriness of their visit to the library. Two basic patterns were revealed in re- gard to this question. First of all, those visiting the library for study purposes felt that their visit, on the average, was slightly less satisfactory than did those who came in search of specific informa- tion, material, or service. It is again pos- sible that the provision of auxiliary study facilities in dormitories and student unions for undergraduates would dimin- ish this dissatisfaction with library facil- ities. Second, though undergraduate stu- dents in all four classifications rated the satisfactoriness of their visits at about the same average value, graduate stu- dents and faculty members, especially those seeking information, materials, or services, were substantially more favor- able in their appraisals. When respondents were classified ac- cording to frequency of previous visits, results were somewhat less clear. While a more favorable average response was obtained from the frequent users who were seeking specific items, no readily explicable trend was apparent among those who were using the library for study purposes. A number of interpreta- tions of the data are possible. The most logical one is that those who visit the library frequently are more familiar with its idiosyncracies and thus more likely to complete their searches of library re- sources successfully. An alternate expla- nation of the results cannot be ignored, however. It may be that the exact oppo- site is true. It is possible that persons who, for one unknown reason or an- other, tend to view the library and its services favorably also tend to use the li- brary more frequently than do those who view it in an unfavorable light. Data were also analyzed for the ques- tion dealing with the library users' eval- uation of the helpfulness and courtesy of the library staff. Responses to this question were obtained only from those individuals seeking information, mate- rials, or services, since it was felt that those individuals coming to the library only for study purposes would rarely have meaningful contact with the library staff. Again the over-all favorableness of the responses was heartening. When the respondents were grouped by university classification, no consistent pattern was revealed, though it did ap- pear that faculty members considered the service rendered somewhat more sat- isfactory than did students. Whether this represents variation in the behavior of the library staff members when in con- tact with faculty members, or a differ- ence in the perception by the faculty respondents, cannot be determined from the results of this study. No consistent patterns were found where responses to this question were classified according to frequency of previous visits. The open-ended question dealing with sources of the respondents' satisfaction or dissatisfaction generated a surprising A Library Looks at Itself I 421 number of highly detailed answers. As might be expected, these ranged from the complimentary to the critical and from the constructive to the sarcastic. It was generally possible to associate these comments with .particular phases of the library's operations, which permitted the discovery of one of the most interesting results. A strikingly consistent set of differences was found between those comments associated with study facilities and those concerning the information, material, or service processing facilities of the library. While the comments asso- ciated with the former dealt almost ex- clusively with the physical environment (temperature regulation, noise, comfort, etc.) , those associated with the latter dealt almost exclusively with the qual- ity and courtesy of the personnel staff- ing the facilities. THE lMPAcr OF THE STUDY · At this writing it is difficult to know the full, long-range impact of the study, since consideration of many of its con- clusions is still in progress. It is already apparent, however, that a number of important benefits have accrued to The Ohio State University libraries as a re- sult of the study. First of all, though the questionnaire will doubtless be refined and improved in any future applications, it has been shown to be an effective tool for the continuing measurement of library-cam- pus relations. Library users are willing and able to provide information about their perceptions of the library and its services. It is possible to design a ques- tionnaire in such a w ay that the infor- mation it provides is specific and con- sistent enough to permit the identifica- tion of particular trouble spots and eval- uation of the effectiveness of programs designed to minimize difficulties . Second, the study provided a body of information about present sources of user satisfaction and dissatisfaction with 422 I College & Research Libraries • November 1967 library service on which such programs might be based. Faculty and students using different areas of the library re- sponded differently to the questions. Variations were found in the ability of groups with different purposes to obtain satisfactory library service. In some re- spects the information uncovered merely served to confirm suspicions based on passive methods of evaluation, but in other cases it gave a picture of the li- brary and its services different from that which would be drawn from the usual run of "gripes" and compliments aimed at the library staff. Third, the study provided the library with data which should be of value in both the long and short-range planning and administration of library and related facilities. This information, particularly that dealing with the use of the library as a general study hall, if confirmed in future studies, could provide the basis for substantial rethinking of the role of the library and that of auxiliary study facilities throughout the university com- munity. The information on variations in usage of library facilities during the day may prove to be an important aid in staffing the various areas and services of the library. In a broader sense, the data on the nature of library users may have the most far-reaching impact. Certain indi- viduals use the library heavily. Others apparently use it hardly at all. Deter- mination of the reasons for these differ- ences may prove to be the key to under- standing where the library is succeeding and where it is failing in its job of dis- seminating information, and thus provide the basis for making it an even more effective force in the total educational program of the university. • •