College and Research Libraries ing one feel the unseen presence of Humpty Dumpty. The discussion on page 6 adduces the properties of stable closed- loop systems to open library systems. On page 8 we are asked to believe that the in- puts to a library system are its goals. Else- where we are told that a library sets its goals, and given a flowchart implying that the setting of goals is a part of action re- search. Although Swisher and McClure don't make common mistakes in their discus- sion of statistical inference, there is still something to be desired. Most library re- searchers today will be presented with SPSS output or something like it. The au- thors could have shown us what that looks like, and have illustrated it with a reasonable set of ample data (perhaps fifty or one hundred data elements.) If their mission is to overcome the fright librarians may feel upon seeing this stuff, the book should display one or two tame examples, to ease that fright. Rather earlier, on page 16 they cite a hy- pothetical case in which a study estab- lishes "a statistically significant relation- ship . . . between women undergraduates and skills taught." I have no idea how the rows and columns of the cross tabulation would be labeled, and I submit that the reader won't either. If the authors do, they should have told us. If they don't, then how can we be confident of their seri- ousness? Furthermore, a key point about the'' use of statistics" is not brought out. The whole idea of confidence intervals is de- signed to prevent premature rejection of some natural hypothesis (usually called the null hypothesis, H 0 ) in favor of an al- ternative that may appear better through the action of chance alone. In very rough language, the 95 percent confidence inter- val is designed to make the odds against this particular mistake 19:1. HOWEVER! In action research we are usually not . ,, testing a new fertilizer'' (perhaps that is more the domain of the reader of type III research)-we are trying to "learn some- thing new.'' Most statistical packages build in the null hypothesis that variables are unrelated. That is absurd. What we usually want to ,know is: "How much are Recent Publications 93 they related?" Is the relation. of manage- rial or economic significance? If I am trying to estimate whether a par- ticular library is circulating as many books as it ought to, I have some idea that this is related to the number of students enrolled in the departments that it serves. To see whether it is "off the line" I assemble the relevant data and draw some kind of plot. If I hand the problem to a statistician she may do a regression analysis, and may tell me that the R-squared value is large, and that I can have high confidence in the re- gression. What that means is that the (ab- surd!!) null hypothesis built into com- puter program (namely that the two variables have nothing to do with each other) can be rejected. It does not mean that every branch ought · to lie on the curve. (This can be dealt with by calculat- ing the band of error, which some pro- grams do, but my point is that we are not interested in preserving the null hypothe- sis here-it is a straw woman.) To sum up, the authors know a great deal about research, and about statistics, but they have not shared the most impor- tant parts of that knowledge with their readers. The project planning chart (page 29) is a useful example for someone who has not done project research before. Chapter 4, on surveys and questionnaires, contains some good tips and pointers. Taken together, the book cannot be rec- ommended. It is not informed by a single critical intelligence, and in places it looks as if the authors shared a single sentence (many run to sixty and seventy words) making the same point twice. The impre- cision in the treatment of ideas will disturb experienced managers and experienced researchers alike. It would make a poor in- troduction to either subject for those with- out experience. In spite of some bright spots, this is rather more a book about the literature than about research. The impor- tant gap is still unfilled.-Paul B. Kantor, Tantalus Inc.~ Cleveland, Ohio . Evaluation of Reference Services. Ed. by Bill Katz and Ruth A. Fraley. New York: Haworth, 1984. 334p. $29.95. LC 84- 12898. ISBN 0-86656-377-6. (This work 94 College & Research Libraries has also been published as The Reference . Librarian, no . 11, Fall/Winter 1984). ''At present, it is still a rare library that has an accurate statistical description of its reference department's quantitative in- put, throughput, or output, let alone its qualitative output." "Unfortunately, li- brarians frequently have little knowledge as to the overall quality of reference ser- vices provided, nor do they engage in an ongoing program of assessment, training, and program development vis a vis refer- ence services.'' ''. . . almost all studies of reference service (as of other areas of li- brary service), have refrained from deal- ing with the benefits of reference service.'' These three representative statements from the introductory sections of three of the twenty-five articles in this collection focusing on the why and how of evaluat- ing reference service show that, while much has been written, relatively little has been achieved in this area. Editor Katz introduces the topic by stat- ing the case for evaluating reference ser- vices. Collectively the rest of the essays present a very fragmented view; at least some of these fragments could have been pieced together had Katz attempted a syn- thetic concluding essay. As it is, various · authors propose various techniques, some for evaluating reference services, some for .evaluating reference librarians, some for evaluating reference tools . Most are mired in the tradition of the relatively young literature evaluation (traced here iP Alvin Schrader's citation study of Ter- rence Crowley's and Thomas Childers' unobtrusive studies of the accuracy of ref- erence service in public libraries in the late 1960s) and discuss only traditional refer- ence service based on the use of print ref- erence tools. Only one article seriously considers online references services; how- ever it also honors the tradii:ion by discus- sing online services as something wholly apart from traditional services. Several of the articles describe the way reference service is evaluated in their au- thors' reference departments-examples of the notorious "how-l-run-my-library- good" genre. This is not to say that these articles are bad; however, they ask the reader to accept their authors' assump- January 1986 tions about what constitutes good refer- ence service or good management. For ex- ample, Margaret A . Joseph's'' Analyzing Success in Meeting Reference Department Management Objectives Using a Comput- erized Statistical Package" simply as- sumes the value of an MBO ·system in an academic library reference department. The same is true of Mignon S. Adams and Blenche Judd's ''Evaluating Reference Li- brarians: Using Goal Analysis as a First Step." Unless one accepts MBO as a valid measure of performance, these articles have very limited value . A lack of clarity about values lies at the .. root of the evaluation problems . It seems that every librarian intuitively knows what constitutes good reference service or a good reference librarian, but none of these authors has been able to articulate that in a meaningful statement with which even a slim majority of the other authors (as well of their colleagues in the field throughout the profession) can agree. Some equate quality to speed of service. Others tout accuracy of factual informa- tion and others the amount of searching the librarian performs for the patrons as the measures of quality. A number of the articles mention, but none attempts to ap- ply, the ALA Reference and Adult Ser- vices Division's " A Commitment to Infor- mation Services: Developmental Guidelines." As the name implies, these are guidelines, many hortatory in nature, not definitive standards by which to mea- sure and evaluate reference service . Evaluation implies measurement as a first step . However one cannot discern in these papers any consensus on what ought to be measured, much less on how to measure. There is also disagreement about who should judge a library's refer- ence service-that library's reference staff, reference staff from another library, or the patrons who receive the service. Because it is the most ''real, '' the last choice has the greatest appeal, but it is flawed by the con- sideration that few patrons operate from a firm enough bibliographic knowledge base to judge reference services well. A concluding essay could not possibly have drawn together and synthesized all of the divergent approaches to a very FOR ALL THOSE WHO MAKE KNOWLEDGE AN OPEN BOOK No small credit for the modern day distribution of information goes to Mr. Gutenberg, a former goldsmith who created the first movable type ... with inspira· tion and infinite attention to detail. · And no small credit goes to today's information manager, who provides ac· cess to the world's knowledge ... with equal professionalism and craftsmanship. When you spend your days making the world an open book, you expect sup- port from professionals who are as meticulous as you are. EBSCO provides that support, with more than 165,000 titles on file-and professional staff providing responsive, personal service. Count on EBSCO to respect the inspiration and attention to detail you re- quire in subscription services. When it comes to serials, we're your open book. ii,tfi•J SUBSCRIPTION SERVICES P.O. Box 1943 Binningham, AJabarna 35201 205-991-1182 --~----------------------~--------------------------------- 96 College & Research Libraries broad topic. Nevertheless, it could have outlined the agenda the profession must follow to evaluate reference services in public and academic libraries. The profes- sion must define what it means by refer- ence services (including online services); it must establish standards for these ser- vices; it must devise techniques for mea- suring services against these standards. It must integrate into a cumulative judg- ment the individual judgments of the vari- ous factors F. W. Lancaster identifies in his article as things that can affect the qual- ity of reference service. These factors in- clude the conduciveness of the environ- ment for information seeking, library policies, the reference collection, library staff, question complexity, the abilities of the user, and the existence of referral agencies. Thus far attempts to evaluate reference service fall well short of this ambitious mark. The best hope is offered by efforts such as the one Marjorie Murfin and Charles Bunge describe in which both li- brarians and patrons in a number of li- braries complete questionnaires describ- ing and evaluating particular reference encounters. All of this data is then ana- lyzed by computer. Although they cau- tion that their results are preliminary and subject to revision after further analysis, one can conclude that good reference ser- vice is labor intensive in that it takes time and that it is more likely to be judged effec- tive if the librarian searches for the infor- mation requested rather than suggests a strategy through which it might be found. Work must continue on this and other techniques until collectively they reach a point at which someone can synthesize them into the best possible way to mea- sure and evaluate reference. Several articles explain how to evaluate databases and reference works and one ar- ticle discusses reference collection poli- cies. Because these articles fail to consider the impact on library patrons, they are pe- ripheral to the volume's central concern of evaluating reference service. Collectively these articles capture the state of the art of evaluating reference- not only the techniques but also the beliefs January 1986 the profession holds about evaluation. Thus far, belief in evaluation's value far outweighs the results derived from evalu- ation experiments. This volume states the problems; it does not offer solutions. However, because methods of evaluating reference service must be found and be- cause this overview comes at a time of re- newed interest in the evaluation issue, it ought to encourage both theoreticians and practitioners to work on the agenda out- lined above.-James R. Rettig, University Library, University of Illinois at Chicago. Campbell, Duncan D. The New Majority: Adult Learners in the University. Edmon- ton, Alberta, Can.: Univ. of Alberta Pr., 1984. 146p. $11.50. LC 84-091063-0 ISBN 0-88864-097-8. Although this book has a Canadian fo- cus, it deals with an important issue of higher education that should be of equal interest in the United States. Campbell, a professor of continuing education and higher education at the University of Al- berta, argues the importance of institu- tions coming to terms with the educa- tional needs of working adults beyond traditional college age. Programs serving this group have frequently been outside the mainstream of normal University pri- orities in both Canada and the United States; but with changing demographic and social patterns, a group once seen as peripheral to the central mission and goals of higher education is now an increasingly important segment of its population. Campbell believes that universities must face this fact and act upon it if they are to remain dominant in the education field. The points raised in this short book are good ones, but one wonders if this was the best means for transmitting them. There is a good deal of repetition between sections and maybe a long, well-written article in a prominent journal would have presented the message more concisely to a broader audience. The first section provides an historical retrospective on continuing education in Canada with reference to other countries, especially the U.S. and the United King- dom. Later sections deal with the rhetoric,