College and Research Libraries Letters To the Editor: It is no easy task to write a letter critical of an article in which one's name is prominently displayed and for the most part generously treated, especially an article whose senior author is a friend and colleague. Neverthe- less, I feel obliged to point out that the article by Larry Hardesty, Nicholas P. Lovrich, Jr., and James Mannon appearing in your Janu- ary 1982 issue ("Library Use Instruction: As- sessment of the Long-Term Effects") is se- verely flawed. I have elsewhere shared my thoughts about the strengths and weaknesses of the twenty-item "skills test" developed at De- Pauw ("Evaluating Bibliographic Educa- tion: · A Review and Critique," Library Trends , Summer 1980) and have no desire to go over that ground here. Nor am I under- taking a systematic assessment of every posi- tive or negative aspect of the article. Rather, I am concerned with what seem to be serious methodological problems. To review briefly: the article describes an attempt "to assess the question of long-term retention of library-use skills," considered by the authors to be "the central question of this evaluation" (p.39). In 1977, just prior to the advent of a bibliographic instruction pro- gram at DePauw, a library-skills test was ad- ministered to a group of freshmen and to se- niors at the institution. Then, in the spring of 1980, the same test was administered to the graduating seniors. Based on the data provided in table 1 of the article (a single average score for the re- spondents in each year is not provided by the authors), my computations are that the 1977 seniors had a mean score of 14.84 items cor- rect on the twenty-item test, while the 1980 seniors scored a mean of 15.81-hardly a stunning improvement. Much more serious, however , is the fact that in their attempt to demonstrate a high correlation between COLLEGE & RESEARCH LIBRARIES amount of exposure to bibliographic instruc- tion and skill possession as measured by the test, the authors include in their analysis 102 students who received bibliographic instruc- tion sometime after the freshman year. (See table 2, "Measures of Exposure.") Some of these students could have received instruc- tion in their senior year, perhaps even shortly before taking the test in the spring of 1980. Thus even if one agrees with the authors that the three years between 1977 and 1980 con- stitute a reasonable definition of "long-term" retention, apparently only a minority of the 230 analyzed responses were individuals who had instruction only as freshmen, and there- fore are the only respondents who can be le- gitimately analyzed for long-term retention. As for the 82 additional respondents ana- lyzed in the panel study, the text on p. 44 im- plies no bibliographic instruction since the freshman year, while table 4 on that page implies that some did receive instruction later. Since table 2 shows the scores of the "Freshman Only" group to be noticeably lower than those of students receiving upper- division instruction, it is quite possible that further analysis would indicate that three years after instruction their performance, on average, was not markedly better than the 1977 seniors who had received no instruc- tion. Hopefully, the authors will provide the data necessary to settle this point. Until they do, their case for retention, the heart of the article, remains undemonstrated. The article's other major problem relates closely to its demonstration that more fre- quent bibliographic instruction is positively correlated with better scores on the skills test and more highly correlated than SAT verbal scores or GP A. As most readers know, how- ever, correlation and causation are not syn- onymous. If at DePauw upper-division bib- liographic instruction is provided when I 353 354 I College & Research Libraries· July 1982 MCGREGOR "PERSONALIZED SUBSCRIPTION SERVICE" Every customer is assigned an experienced "Home Office" representative. You correspond direct; any title needs, changes, cancellations or problems can be handled promptly by letter or phone. This makes your job easier and keeps you abreast of your subscription needs at all times. With over 45 years experiWJce, McGregor has built a reputation of prompt and courteous service on both domestic and International titles. We prepay subscrip- tions ahead of time. Our customers, large and small, like the prompt attention we give them. We think you would tool Ask about McGregor's "Automatic Renewal" plan de- scribed in our new brochure. Wrltl today for your free copy. OUR 49th YEAR Mount Morris, llllnolsl1054 Engineering Index publications. Abstracts of virtually every important article published in every engineering discipline. Last month. Last year. Or as far back as 1884. Call toll-free or write for copies of our folder ''A Complete Guide to Engineering Index Publications," yours to d istribute free. Engineering Information, Inc. 345 East 47th Street New York, New York 10017 Call toll-free 800-221-1044 In New York State, call212-705-7615 students have library-related assignments and is generally not provided when they do not, it is not at all surprising that more instru_ctionllibrary use will yield higher scores on a library skills test. Demonstrating that the key variable is bibliographic instruc- tion rather than library use itself would require an experimental design, testing whether the scores of individuals known to have undertaken significant library use with- out instruction were lower than instructed students, thereby "controlling for" library use. This the authors have not done. The authors and I agree that statistical techniques often have an important role to play in evaluation, including the evaluation of bibliographic instruction . But those tech- niques must flow from a sound design. Ma- chines and machine analysis are no substitute for people asking the right questions. - Richard Hume Werking , Associate Director, Collection Development, Assistant Professor of History , Trinity University , San Antonio , Texas. Publishers of Engineering lndex8 Monthly, Engineering Index Annual, Energy Abstracts. Bioeng ineering Abstracts, and Compendex 8