College and Research Libraries By M O R R I S A. G E L F A N D Techniques of Library Evaluators in the Middle States Association THIS STUDY* concerns itself primarily with the techniques currently used by library evaluators in the M i d d l e States Association of Colleges a n d Secondary Schools. It deals also with library evaluation, general institu- tional evaluation, a n d accreditation by the Association in order to provide a p p r o p r i a t e , indeed necessary, b a c k g r o u n d for better u n d e r s t a n d i n g of the task of the library evaluator. P R O C E D U R E T o study the techniques of library eval- uators a comprehensive q u e s t i o n n a i r e was designed for submission to those who have served the M i d d l e States Association d u r i n g the past several years. T h e q u e s t i o n n a i r e (a copy of which may be borrowed from the author) was organized u n d e r three p r i n c i p a l headings which rep- resent the usual stages of a library evalua- tion for the M i d d l e States Association: (1) p r e p a r a t i o n for an evaluation visit; (2) the evaluation visit; a n d (3) p r e p a r a t i o n of the evaluation report. Q u e s t i o n s u n d e r each of these headings were p r e p a r e d on the basis of the writer's personal experience on M i d d l e States library evaluations since 1949, study of pertinent M i d d l e States Association pub- lications, a n d a f t e r reference to Wilson a n d T a u b e r , 1 M c D i a r m i d , 2 a n d Lyle. 3 Many questions r e q u i r e d only a " Y e s " or " N o " answer, but some relating to specific tech- niques or practices called for an indication of the degree to which a particular method * Paper presented at the Eastern College Librarians Conference, Columbia U n i v e r s i t y , November 30, 1957. 1 Louis R. Wilson, and Maurice F. Tauber, The University Library. 2nd ed. ( N . Y . : Columbia Uni- versity Press, 1956). 2 E. W . McDiarmid, Jr., The Library Survey (Chi- cago: A L A , 1 9 4 0 ) . 3 Guy R. Lyle, The Administration of the College Library. 2nd ed. rev. ( N . Y . : H. W . W i l s o n , 1949). Mr. Gelfand is Librarian, Queens Col- lege, Flushing, New York. or device was used, u n d e r the headings " R e g u l a r l y , " " O c c a s i o n a l l y " a n d " N e v e r . " R e s p o n d e n t s were also invited to submit comments where they considered them ap- p r o p r i a t e in order to provide a better view of their practices. A " g e n e r a l " section was a d d e d to the q u e s t i o n n a i r e for the p u r p o s e of eliciting information concerning the back- g r o u n d of the evaluators, the benefits, if any, which they received as library evalua- tors, a n d their general comments on the evaluation process. Mr. F. T a y l o r Jones, executive secretary of the M i d d l e States Association Commis- sion on Institutions of H i g h e r E d u c a t i o n , s u p p l i e d a list of twenty-nine librarians who were " a m o n g the librarians who have worked successfully with the M i d d l e States Associa- tion teams in the past several years." 4 T o this g r o u p two names were a d d e d by the writer a n d the questionnaire was then mailed to thirty-one persons. Twenty-nine responses were received, representing 93 per cent of the persons who were questioned. B e a r i n g in m i n d the limitations of the ques- tionnaire m e t h o d , some ambiguities in the q u e s t i o n n a i r e used in this study, instances where no responses were m a d e to specific questions, a n d the relative size of the g r o u p to which the q u e s t i o n n a i r e was directed, it is believed that the responses as a whole pre- sent a valid picture of current techniques most commonly used, a n d give some indica- tion of the extent of their use. For presentation of M i d d l e States Associa- tion policies a n d procedures relating to ac- creditation a n d library evaluation, extensive use has been m a d e of official publications of the Association. T o avoid errors in inter- pretation a n d to state the position of the As- sociation as fully as a p p e a r s necessary, the writer will q u o t e liberally from these pub- lications. 4 F. Taylor l o n e s , Letter to M. A. Gelfand, of Octo- ber 30, 1957. JULY 1958 305 L i b r a r y evaluations for accrediting pur- poses present peculiar p r o b l e m s which need to b e clarified a n d solved. T h i s condition has been recognized a n d expressed frequent- ly. Only recently in a p a p e r presented at a m e e t i n g of the A C R L J u n i o r College Sec- tion in Chicago, F e b r u a r y 1, 1956, Dr. Man- n i n g M . Pattillo, formerly associate secretary of the C o m m i s s i o n on Colleges a n d Univer- sities, N o r t h C e n t r a l Association, said: A f t e r some seven years of c o p i n g with the practical p r o b l e m s of e x a m i n i n g in- stitutions for accrediting purposes, I can testify that the library is o n e of the most difficult phases of an institution's p r o g r a m to evaluate effectively. T h i s is generally recognized a m o n g m e n who h a v e m a d e m a n y surveys of colleges a n d universities. I n almost every other area of the institu- tion's p r o g r a m i n f o r m e d persons have a reasonably clear notion of what to look at a n d how to draw conclusions in an eval- u a t i o n . . . . In the area of the library there is n o such unanimity . T h e r e seem to be serious problems in almost every m e t h o d of a p p r a i s i n g the effectiveness of a college library. B e y o n d certain very general prop- ositions which w o u l d be widely accepted, there is a paucity of constructive thought as to how to proceed in the specific situa- tion. Somehow we need to d e v e l o p some very different way of l o o k i n g at the whole p r o b l e m . 5 Pattillo's remarks have equal, if not great- er, pertinence to the p r o b l e m s of e v a l u a t i n g university libraries. I n a b r o a d sense all types of library eval- u a t i o n s have a c o m m o n p u r p o s e : to deter- m i n e how effectively the library s u p p o r t s the e d u c a t i o n a l p r o g r a m of the institution it serves. B u t library evaluations by regional accrediting agencies differ from other types in that they are c o n d u c t e d as an integral p a r t of total institutional evaluation with the primary objective of d e t e r m i n i n g how successfully the institution is m e e t i n g its self- declared p u r p o s e s a n d objectives. H i g h l y de- tailed descriptions a n d analyses of library re- sources, services, a n d facilities are not, a n d usually cannot, be m a d e by the library eval- u a t o r of the accrediting association. Efficien- cy a n d economy of library services are not primary subjects of inquiry as they might b e 6 M. M. Pattillo, "The Appraisal of Junior Colleges and College Libraries." College and Research Librar- ies. X V I I ( 1 9 5 6 ) , 397-402. ill a conventional library survey. T h e y are significant only in so far as their absence in- dicates that the library is not fulfilling its its mission. T h i s is not to say that improve- ments in library p e r f o r m a n c e have not re- sulted f r o m the accreditation process. It is p r o b a b l e that some libraries have been im- p r o v e d very substantially as a result of searching self-surveys or surveys by outsiders which have been conducted in p r e p a r a t i o n for an evaluation by a regional accrediting agency. It is also p r o b a b l e that improve- ments in the quality a n d efficiency of library services have resulted f r o m the a d o p t i o n of r e c o m m e n d a t i o n s m a d e by visiting teams. M E A N I N G O F M S A A C C R E D I T A T I O N T h e M i d d l e States Association represents a voluntary association of higher institutions — a m u t u a l aid society—whose p u r p o s e s are " T h e i m p r o v e m e n t of educational p r o g r a m s a n d facilities a n d the b r o a d e n i n g of educa- tional o p p o r t u n i t y . M e m b e r s h i p in the As- sociation is synonomous with accreditation by i t . " Accreditation by the M i d d l e States As- sociation indicates that an institution has been f o u n d qualified for m e m b e r s h i p in the Association a f t e r evaluation by its own staff a n d by a team of qualified colleagues f r o m other institutions. E v a l u a t i o n f o r M i d d l e States member- ship covers the entire institution, includ- ing all the instructional a n d non-instruc- tional activities of every constituent part a n d unit. M i d d l e States accreditation also e x t e n d s to the whole institution. Accreditation signifies that the institu- tion offers c o m m e n d a b l e p r o g r a m s l e a d i n g to the achievement of its own p a r t i c u l a r objectives. It indicates that all its work is conducted at a satisfactory level, in the j u d g m e n t of the M i d d l e States Associa- tion, but not that it is all necessarily of u n i f o r m quality. 6 T h e last phrase of the p r e c e d i n g sentence has special significance for libraries. I t is q u i t e conceivable that the library of a given institution might not be r e g a r d e d as one of its strongest features a n d that this institu- tion would nevertheless be accredited. Of- ten, however, it is f o u n d that where the li- 6 Middle States Association of Colleges and Second- ary Schools. Commission on Institutions of Higher Education. Middle States Membership and Accrediation. Document No. 3.12:1, September, 1955. 306 COLLEGE AND RESEARCH LIBRARIES brary is i n a d e q u a t e there are other signif- icant weaknesses which, taken together, tend to s u p p o r t a j u d g m e n t of overall institu- tional weakness. T h e Association makes certain formal stipulations as to the kinds of institutions eligible for membership. In a d d i t i o n to these f o r m a l stipulations, the Association holds a concept of the es- sential n a t u r e of higher education which, without d i s p a r a g e m e n t of other worthy kinds of instruction, enters into its deter- m i n a t i o n of eligibility, although its fac- tors have to be tested in the evaluation itself. T h e y are: T h e extent to which the institution's curricula provide, emphasize, or rest u p o n general or liberal education. T h e e x t e n t to which its objectives a n d p r o g r a m s seek to inculcate power to form i n d e p e n d e n t j u d g m e n t , to weigh values, a n d to u n d e r s t a n d f u n d a m e n t a l theory, rather than solely to amass facts or acquire skills. Whether its students are stimulated to continue a n d b r o a d e n their education be- yond the p o i n t they must reach to earn its credits, certificates, or degrees. 7 T h e library implications of this concept are clear. T h e y have been recognized in pro- fessional library p u b l i c a t i o n s a n d are reflect- ed in the attitude of the M i d d l e States As- sociation toward library evaluations. T h e M i d d l e States view of accreditation rests u p o n the premise that the impor- tance of accreditation is its effectiveness as a stimulant to educational improve- ment, a n d that the process leading to ac- creditation must accordingly be designed to be of m a x i m u m service to the facultv, administration, a n d trustees of the institu- tion concerned, rather than to the accred- iting agency. T h e M i d d l e States Association holds that each institution must be j u d g e d in reference to its own declared p u r p o s e s a n d objectives; that the j u d g m e n t should be m a d e jointly by its own personnel a n d competent colleagues from neighboring, b u t not competing, colleges a n d univer- sities; a n d that the significant criteria for the judgment are qualitative. Essentially, therefore, a M i d d l e States evaluation is concerned with (a) the in- stitution's explicit definition of its own task, a n d the adequacy of that definition; 7 Ibid.., p. 2. (b) its plans, resources, a n d procedures for fulfilling its responsibilities; a n d (c) its success in d o i n g so. 8 T H E E V A L U A T I O N P R O C E S S T h e evaluation process begins with a re- quest for an evaluation from a n institution which desires m e m b e r s h i p in the M i d d l e States Association. Or it may begin with a decision by the Association to re-evaluate a m e m b e r institution. In recent years the As- sociation has introduced the policy of peri- odic re-evaluations of m e m b e r institutions on a ten-twelve year cycle. A f t e r a prelim- inary visit to the institution by the Exec- utive Secretary of the M i d d l e States Associa- tion, a tentative d a t e is set for an evaluation visit to take place at least a year later. T h e institution then embarks u p o n what has been characterized as the most v a l u a b l e p a r t of the evaluation process, self-evaluation of its p u r p o s e s a n d objectives a n d of the success it has h a d in achieving them. A t this stage the institution is usually g u i d e d by the ques- tionnaires which the Association has de- signed for institutional self-evaluation a n d by other pertinent publications of the As- sociation. It is also privileged to seek the help of Association officials a n d to turn to outside consultants for assistance in special areas. E v a l u a t i o n teams may r a n g e f r o m five to six m e m b e r s for a small single-purpose in- stitutions, to fifty or mo re for a large, com- p l e x institution such as a university. Li- brarians are always members of these teams, a n d in some instances there m a y be as many as three or f o u r librarians on a large team. M e m b e r s of the e v a l u a t i n g team receive copies of the self-evaluation r e p o r t of the institution for study some time, usually sev- eral weeks, b e f o r e the visit is scheduled to start. T h e n comes the visit which usually takes three days. T h e visit is followed by a r e p o r t f r o m the evaluation team to the Commission o n In- stitutions of H i g h e r E d u c a t i o n of the Mid- dle States Association. T h i s , together with the self-evaluation report of the institution a n d a verbal r e p o r t f r o m the chairman of the evaluation team, is presented to the Commission a n d provides the »basis for its action. T h e chairman of the Commission 8 Loc. cit. JULY 1958 307 then notifies the h e a d of the institution of the Commission's action, a n d the r e p o r t of the evaluation team is m a i l e d to the institu- tion. ACCREDITATION DECISIONS Accreditation decisions are m a d e by the Commission on Institutions of H i g h e r Ed- ucation of the Association. T h e decisions of this g r o u p , which is elected by the member- ship of the Association, can be any o n e of five, r a n g i n g from accreditation of a non- m e m b e r a n d re-affirmation of the accredita- tion of a member, to denial of accreditation to a non-member or d r o p p i n g of a m e m b e r from the accredited list. In recent years, the Commission has qualified some decisions by requesting progress reports in r e g a r d to the i m p l e m e n t a t i o n of r e c o m m e n d a t i o n s m a d e in the evaluation r e p o r t a n d a d d i t i o n a l fol- low-up visits to the institution to ascertain whether certain weaknesses have been cor- rected. In some instances it has r e q u i r e d a full or partial re-evaluation of an institution. T H E W O R K OF EVALUATORS In a d o c u m e n t entitled Your Work as an Evaluator; Suggestions for Team Members, the M i d d l e States Association emphasizes the i m p o r t a n c e of the self-evaluation study of the institution to be visited a n d suggests that it is the "first responsibility [of the eval- u a t o r l to master and think a b o u t its con- t e n t s . " 9 Some a d d i t i o n a l excerpts f r o m this p u b l i c a t i o n reveal clearly the Association's concern for objective, impartial, a n d con- structive attitudes a m o n g evaluators. T h e team's f u n c t i o n is to m a k e a n in- d e p e n d e n t analysis, for the institution's use, of the quality of its per f or manc e, a n d of the effectiveness of its procedures a n d the adequacy of its resources for continual- ly i m p r o v i n g its p e r f o r m a n c e . Your task is not to " i n s p e c t " the institu- tion. W e have no Formulas to give, no rules to a p p l y or patterns to impose. You g o as a colleague, to h e l p identify the in- stitution's strengths a n d discover how to solve its most critical problems. Y o u have been selected because the Commission be- lieves that you, s u p p o r t e d by your team- mates, are competent to d o that. B u t ap- 9 Middle States Association of Colleges and Sec- ondary Schools." Commission on Institutions of Higher Education. Your Work as an Evaluator; Suggestions for Team Members. Document No. 2.41:1, January, 1956. proach the task humbly. N o one knows all the answers. Your a d v a n t a g e lies sim- ply in your detached position. T h e primary consideration to k e e p in m i n d (during the p r e p a r a t i o n of your re- port) is that in it you are s p e a k i n g to the institution just as surely as if you were ad- dressing its assembled staff in person. T h e M i d d l e States Commission, a n d any other agencies which may be o p e r a t i n g in the evaluation, also use the report, but it is designed for a n d directed to the institu- tion. T h a t fact determines its n a t u r e a n d con- tent. Your task is not to describe; ijr is to evaluate. T h e institution's o w n ' s t a f f has described it. You d o n o t need to tell them what they already know. Your p a r t is to assess its work, sympathetically, critically, a n d constructively. 1 0 , T h e M i d d l e States Association concept of the place of the library in a higher institu- tion has been set forth in a remarkably con- cise a n d p r o f o u n d statement recently issued by the Association as an official d o c u m e n t . Designed primarily " f o r the use of faculties a n d evaluation t e a m s , " 1 1 it can be studied with profit by all who are interested in the i m p r o v e m e n t of libraries in higher institu- tions even though some may wish to take issue with statements m a d e in it. Some ex- cerpts from this d o c u m e n t are given here by way of b r i n g i n g to a close the presenta- tion of the M i d d l e States Association's poli- cies a n d procedures a n d p r e p a r i n g the way f o r consideration of the techniques of li- brary evaluators. T h e primary characteristic of a good academic library is its complete identifica- tion with its own institution. T h e meas- ure of its excellence is the extent to which its resources a n d services s u p p o r t the in- stitution's objectives. Every library must therefore be eval- uated in its own setting rather than by comparison with general patterns or norms, because each library must s u p p o r t a particular educational p r o g r a m . T h e prerequisite for library evaluation, accord- ingly, is an exact description of the in- stitution's mission a n d of the m e a n s by which the institution proposes to fulfill it. Given that, scholars can identify the re- 10 Loc. cit. 11 Middle States Association of Colleges and Sec- ondary Schools. Commission on Institutions of Higher Education. Evaluating the Library; Suggestions for the Use of Faculties and Evaluation Teams. Document No. 4.81, October, 1957. 308 COLLEGE AND RESEARCH LIBRARIES sources they must have to accomplish the task. T h e evaluation of the library can then begin. T h e process can be summarized in a series of questions: W h a t access to the world's intellectual a n d creative resources does this institution's educational p r o g r a m require? T o what extent are these re- sources now a v a i l a b l e a n d accessible through the library? H o w can their avail- ability a n d accessibility be increased a n d their use encouraged? Obviously no o n e person can m a k e an evaluation of this kind, nor can profes- sional librarians d o it alone. Identification a n d a p p r a i s a l of the materials to s u p p o r t instruction a n d stimulate research in a given field d e m a n d a scholar's knowledge plus a librarian's skill. Analysis of the use which students a n d faculty are m a k i n g of the library a n d the reasons for it is a i d e d by intelligently designed statistical records, but goes beyond them into educational philosophy a n d teaching methods. L i b r a r y specialists can organize a n d give technical competence to such a study, but must de- p e n d on their professional colleagues to identify the resources r e q u i r e d to meet the full needs of u n d e r g r a d u a t e students, the basic needs of advanced students in each field in which the institution offers such instruction, the professional a n d research requirements of the faculty, a n d for esti- m a t i n g the strength of the collection as it stands. T h e librarians who work with them must g u a r d b a l a n c e a n d coverage, which m e a n s they must have a g o o d gen- eral knowledge of the scholar's field a n d of the teacher's problem. T h e s e considerations suggest the char- acteristics of a g o o d librarian. H e a n d his professional colleagues are responsible for the administration a n d d e v e l o p m e n t of a m a j o r element in the institution's intel- lectual life. H e needs the skill to direct a highly complex organization well, b u t his thinking a n d p l a n n i n g must be that of a teacher a n d scholar, not a curator or technician. H e must be chosen with the same care a n d u n d e r many of the same criteria as other high-ranking faculty mem- bers. Professional qualifications are not enough. H e must know what scholarship is a n d what teaching entails. H e must demonstrate the competence to merit the respect of his colleagues as an educator a n d be given the status that will e n a b l e him to speak with equal voice in their company. H e a n d his professional assist- ants must have an effective place within the faculty councils in order to relate the library properly to the curriculum a n d to ensure g o o d c o m m u n i c a t i o n s in both di- rections. L i b r a r y evaluation involves a study of faculty attitudes a n d teaching methods. T h e faculty is deeply concerned with the library; it is of primary importance in their instructional p r o g r a m a n d in their professional growth. T h e y ought therefore to have an i m p o r t a n t voice in determin- ing its objectives a n d a constant advisory relationship to the h e a d librarian, al- though he should r e p o r t in his administra- tive capacity to the president or d e a n . A n alert faculty never allows a library to suffer from neglect or to diverge f r o m the educational p r o g r a m . N e i t h e r does a g o o d faculty m e d d l e with internal admin- istration or a t t e m p t to deal with technical details—it p a r t i c i p a t e s in establishing ob- jectives a n d general policies a n d expects the librarian a n d his staff to give them effect. T h e faculty usually operates through a s t a n d i n g committee of which the librarian is a member, p e r h a p s secretary, which meets regularly in a n advisory capacity, keeps itself thoroughly info rmed, g u a r d s a n d advances library interests, a n d reports frequently to the faculty for discussion, counsel, or confirmation. T h e i m p o r t a n c e which this committee's work can have for the institution warrants selecting its mem- bers with great care a m o n g those who are most interested in the library, use it ex- tensively themselves, a n d u n d e r s t a n d the difference between advisory a n d adminis- trative functions. T h e use the students m a k e of the li- brary—the u l t i m a t e test of its effective- ness—is not an accident. It is the result of many forces, including habit, convenience, the ready availability of the materials the students want, the attractiveness of the setting, staff personalities, a n d the way the librarians a n d instructors work together. B u t it is chiefly the result of the faculty's teaching methods. If statistics or observa- tion suggest that the library may not be serving as fully as it might or is b e i n g used as a study hall with books from out- side, look for lecture-textbook or other u n i m a g i n a t i v e teaching. Since the quality a n d a m o u n t of library use is o n e of the clearest indices of the k i n d of teaching the students are getting, experienced eval- uators are a p t to turn quickly from the li- brary to the classroom. T h e y know that a s t i m u l a t i n g instructor creates an i n q u i r i n g JULY 1958 309 student, who develops resourcefulness be- cause he wants mo r e than r o u t i n e methods will give him. T h u s g o o d teaching a n d g o o d l i b r a r i a n s h i p u n i t e to p r o d u c e skilled, self-reliant, h a b i t u a l library users. I n d e p e n d e n t a n d honors work p r o v i d e an especially f a v o r a b l e climate for it. Clearly, therefore, the emphasis in eval- u a t i n g a library should be on the appro- priateness of the collection for the in- structional a n d research p r o g r a m s of the students a n d faculty, its adequacy in breadth, d e p t h , a n d variety to stimulate both students a n d faculty, its accessibility, i n c l u d i n g p r o p e r cataloging, the compe- tence a n d interest of the staff, a n d a b o v e all, what h a p p e n s in the r e a d i n g a n d ref- erence rooms. Statistical comparisons need to be h a n d l e d with caution. Percentages of the e d u c a t i o n a l b u d g e t spent on library service a n d growth, per c a p i t a expendi- tures, n u m b e r of volumes, circulation fig- ures, a n d the r a t i o of staff to students a n d of students to seats o f t e n p r o v i d e sugges- tive leads, b u t they should b e studied in context a n d perspective. W h e n the institution's objectives a n d its curricula have been analyzed a n d the resources a n d services the library ought to p r o v i d e to s u p p o r t them have been de- scribed, questions like the following may clarify the final stages of the p r o b l e m . Others will suggest themselves to the eval- uators. T h e y must b e dealt with candidly a n d objectively, of course, a n d every neg- ative answer should b e c o u p l e d with a practicable r e c o m m e n d a t i o n . 1 2 H e r e follow a g r o u p of twenty-six ques- tions which deal with the m a j o r character- istics of g o o d library service. T w o of these questions are q u o t e d to p r o v i d e a n indica- tion of their searching quality: 1. Is the library book stock sufficiently b r o a d , varied, authoritative, a n d u p to d a t e to s u p p o r t every p a r t of the under- g r a d u a t e instructional p r o g r a m ? 2. Is there a d e q u a t e a d d i t i o n a l strength in source, m o n o g r a p h i c , a n d periodical material f o r any g r a d u a t e work, honors work, a n d research which is offered or p r o p o s e d ? " 1 3 T E C H N I Q U E S O F M S A L I B R A R Y E V A L U A T O R S T h e results of the q u e s t i o n n a i r e inquiry will be presented u n d e r these h e a d i n g s : (1) 12 Loc. cit. 13 Loc. cit. p r e p a r a t i o n for an evaluation visit, (2) the e v a l u a t i o n visit, (3) p r e p a r a t i o n of the eval- u atio n report, a n d (4) qualifications of li- brary evaluators a n d their comments on the evaluation process. R e s p o n s e s will be col- lated a n d analyzed a n d such conclusions as arise from the d a t a will be presented. In ad- dition, the writer will m a k e some personal observations based u p o n his experience a n d present r e c o m m e n d a t i o n s . It should be em- phasized that the analysis of d a t a will be based only u p o n the replies which were re- ceived. While no claims are m a d e for the statistical reliability of the conclusions which will be presented, it would a p p e a r that a representative g r o u p of qualified evaluators has r e s p o n d e d to the questionnaire. T h e i r views a n d practices as presented a n d inter- p r e t e d here may therefore be characterized as adequately representative. P R E P A R A T I O N F O R A N E V A L U A T I O N V I S I T A f t e r a librarian accepts an invitation f r o m the executive secretary of the M i d d l e States Association to serve on a particular evaluation team, he will usually receive from the M i d d l e States Association a list of the members comprising the whole team a n d an " E v a l u a t i o n H a n d b o o k , " a collection of official p u b l i c a t i o n s of the M i d d l e States As- sociation which relate to the aims, policies, a n d practices of the Association. T h e con- tents of this h a n d b o o k will vary f r o m time to time as new p u b l i c a t i o n s a p p e a r or o l d ones are superseded. For the novice eval- uator, the h a n d b o o k is the most v a l u a b l e single source of i n f o r m a t i o n a b o u t the p o i n t of view a n d methods of the Association. Ex- perienced evaluators also find the h a n d b o o k to be v a l u a b l e for review p u r p o s e s a n d a way of k e e p i n g u p to d a t e with new publica- tions of the Association. Several weeks b e f o r e the visit is scheduled, each of the evaluators usually receives a copy of the complete self-evaluation r e p o r t of the institution which is to be evaluated, together with catalogs, a n d other materials such as bylaws, p r o m o t i o n a l literature, etc. O f t e n the report is a long a n d a p p a r e n t l y formi- d a b l e d o c u m e n t or a series of documents de- p e n d i n g u p o n the size a n d complexity of the institution to be evaluated. As he studies the self-evaluation r e p o r t , the evaluator may encounter a m b i g u o u s pas- 310 COLLEGE AND RESEARCH LIBRARIES sages and decide to write to the librarian of the institution for clarification or additional information. H e may also find it advisable to consult professional library literature for assistance in interpreting or evaluating the report. At this point in his preparation the evaluator may wish to make some plans for the visit, note subjects he will want to in- quire into further during the visit, and list a few questions which arise from his reading. T h e questionnaire inquiry revealed that nearly all respondents find it helpful to re- view regularly the Middle States Association publications relating to policies and proce- dures in preparation for a visit. T h e self-evaluation report of the institu- tion under study is without doubt a major element in the evaluation process. How many library evaluators read this report in its entirety; how many selectively? T h e data in T a b l e I, reveal that most li- brary evaluators read the entire report of a single-purpose institution, while a m o n g those who must evaluate a complex institu- tion, the a p p a r e n t practice is to read the report selectively. In this connection, selec- tively, is taken to mean that the library sec- tions alone are read or the library and a few additional sections concerning subjects close- ly related to the library are read. It is ap- propriate to mention here that only fourteen out of the twenty-nine respondents h a d eval- uated complex institutions; fifteen h a d never done so. Evaluators of complex institutions, however, often are invited also to evaluate single-purpose institutions, and they, there- fore, account for some of the responses re- corded for single-purpose institutions, while some library evaluators who have visited only single-purpose instiutions indicated what they would do with the report of a complex institution. Twenty-three library evaluators indicated that they d o not correspond in advance with the librarian of the institution to be visited. Six stated that they write to the librarian before the visit but one commented " n o t always." O n e evaluator commented that he "would not consider this [writing, that is] advisable practice." In answer to the question: Do you make use of books and other materials to assist you in assessing the report of the institution to be visited and in locating problem areas? T A B L E I SELF-EVALUATION R E P O R T OF INSTITUTION REPORT READ TYPE OF INSTITUTION ENTIRELY SELECTIVELY S i n g l e - p u r p o s e i n s t i t u t i o n C o m p l e x i n s t i t u t i o n 2 5 4 2 13 twenty-four replied in the affirmative; four, negatively; and one did not reply. T h e ma- terials most commonly used, although not always regularly, are listed in T a b l e II. It is interesting to note here the extent to which the annual statistical summary in the J a n u a r y issue of CRL is used by library evaluators, a n d that the use of American Li- brary Association, Classification and Pay Plans14 is not insignificant. Most of the evaluators queried (twenty- five) indicated that they formulate a plan for the projected visit during the prepara- tion period. B u t two evaluators stated that their plans at this early stage were quite gen- eral. One respondent said: "Generally [yes] but more specifically at the first evaluation meeting," referring to the initial meeting of the visiting team on the campus of the institution undergoing evaluation. Another reply was along similar lines: "Yes, only in general terms. I find it better not to make too formal plans. Each institution is u n i q u e and the atmosphere of the institution fre- quently gives the surveyor ideas on the scene." T h e complete catalog of the institutions to be visited is read by nineteen of the re- spondents before the evaluation visit. A m o n g the ten remaining evaluators some apparent- ly do not read the catalogue at this time, but the figures are inconclusive as the ques- 14 American Library Association. Board on Salaries Staff, and Tenure. . . . Classification and Pay Plans for Libraries in Institutions of Higher Education. 2nd. ed. (Chicago: ALA, 1947). Library Score Card. Vol. II—Degree-Confer- ring Four-Year Institutions. Supplement to Classifica- tion and Pay Plans for Libraries in Institutions of Higher Education. (Chicago: ALA, 1950). Library Score Card. Vol. Ill—Universities. Supplement to Classification and Pay Plans for Li- braries in Institutions of Higher Education. (Chicago: ALA, 1950.) JULY 1958 311 T A B L E VIII M A T E R I A L S U S E D D U R I N G P R E P A R A T I O N F O R A N E V A L U A T I O N V I S I T M A T E R I A L S USED R E G U L A R L Y O C C A S I O N A L L Y NEVER W i l s o n a n d T a u b e r , The University Library 3 13 4 L y l e , The College Library 9 11 1 M c C r u m , Estimate of Standards 2 9 6 B r a n s c o m b , Teaching With Books 3 10 3 A L A Classificatioii and Pay Plans 6 10 2 College and Research Libraries, J a n u a r y i s s u e 14 9 M c D i a r m i d , Library Survey 1 6 9 O t h e r (by r e s p o n d e n t s ) : H i s t o r y of i n s t i t u t i o n 1 T a u b e r , Technical Services in Libraries 1 M S A , E v a l u a t i o n r e p o r t of o w n i n s t i t u t i o n 1 tion relating to catalogs was not well phrased. A m o n g those who read the catalog selectively, the sections dealing with the aims of the institution, the library, the facul- ty, curricula and course descriptions a p p e a r to be read most frequently a n d in the order indicated. T H E E V A L U A T I O N V I S I T T h e section of the questionnaire which deals with the evaluation visit was designed in the form of a check list. In the first col- umn at the left side of each page, under the heading Methods and Devices, there were listed sixty different methods and de- vices which evaluators use in greater or lesser degree during the course of an evalu- ation. T h e s e included many which are men- tioned in Middle States Association publica- tions, in Wilson and T a u b e r , 1 5 L y l e , 1 6 and M c D i a r m i d , 1 7 some which the writer has used in his work, some suggested by col- leagues. T h e object of this section was to determine what devices and methods are most commonly employed, the degree to which they are employed, and the principal purposes they are designed to serve. T e n columns were arranged to the right of the column listing methods a n d devices to permit the respondent to indicate under each heading the extent, if any, to wh^ch he employed a particular approach. T h e s e head- ings were (1) program of the library; (2) adequacy of library collection; (3) quality 15 Wilson and Tauber. op. cit. 16 Lyle, op. cit. 17 McDiarmid, op. cit. of readers' services; (4) quality of technical services; (5) student use of the library; (6) faculty use of the library; (7) adequacy of space (building); (8) adequacy of library staff; (9) attitude of institution toward li- brary; a n d (10) overall effectiveness of the library. U n d e r each of these headings the respondent could make a check mark in the a p p r o p r i a t e column to indicate whether he used a particular method or device reg- ularly, occasionally, or never. When the responses were tabulated, it was f o u n d that some evaluators had a d d e d a de- vice or two of their own to those already list- ed a n d that some did not respond to every item listed. On the whole, however, the re- sponses to this section of the questionnaire were sufficiently full to justify a full tabula- tion a n d analysis. An analysis of the data was m a d e along two lines. In the first, the object was to deter- mine the relative ranking, if any, of the ten principal headings under which evalua- tion was being conducted. In the second, the object was to discover how frequently each of the methods and devices listed and those that were a d d e d by respondents was , employed. T o determine relative ranking of subjects of evaluation, such as adequacy of the book collection, quality of technical services, a n d the others, the check marks in each of the ten columns headed by these subjects were added together under each sub-heading, that is under the sub-headings, " R e g u l a r l y , " "Oc- casionally," and " N e v e r . " T h e results, which a p p e a r in T a b l e I I I relate only to the re- 312 COLLEGE AND RESEARCH LIBRARIES T A B L E I I I R E L A T I V E R A N K I N G O F S U B J E C T S O F E V A L U A T I O N 1 2 3 4 5 6 7 DEVICES DEVICES DEVICES R E G U L A R L Y O C C A S I O N A L L Y N E V E R S U B J E C T O F E V A L U A T I O N USED R A N K USED R A N K USED RANK N U M B E R N U M B E R N U M B E R O F C H E C K S O F C H E C K S O F C H E C K S Program of library 701 1 309 1 299 1 Overall effectiveness of library 585 2 231 4 255 2 A d e q u a c y of library c o l l e c t i o n 581 3 268 2 241 3 S t u d e n t u s e of t h e library 493 4 212 6 234 4 Q u a l i t y of reader's services 472 5 248 3 219 5 Faculty use of t h e library 471 6 190 7 210 6 A t t i t u d e of i n s t i t u t i o n toward library 465 7 231 5 209 7 A d e q u a c y of space: b u i l d i n g 449 8 188 8 205 8 A d e q u a c y of library staff 449 8 188 8 192 9 Q u a l i t y of technical services 367 9 177 9 143 10 T o t a l n u m b e r of check marks 5,033 2,242 2,207 P e r c e n t a g e of total responses 53 24 23 sponses of the particular g r o u p who an- swered the q u e s t i o n n a i r e a n d are not offered as universally representative indications of the relative importance of these subjects to library evaluators. Fifty-three per cent of the total responses to this section of the q u e s t i o n n a i r e were given u n d e r the s u b h e a d i n g " R e g u l a r l y " in each c o l u m n ; 24 per cent u n d e r "Occasion- ally," 23 p e r cent u n d e r " N e v e r . " As 77 per cent of the responses indicated that the ten subjects of evaluation were considered in greater or lesser degree, it was decided that it w o u l d b e h e l p f u l to c o m p a r e the responses u n d e r " r e g u l a r l y " with those u n d e r "oc- casionally" for some positive indication of relative r a n k i n g a m o n g the subjects of eval- uation. C o m p a r i s o n reveals a fairly consist- ent relationship between the subjects of e v a l u a t i o n ; whether particular methods or devices were employed regularly or occasion- ally, the relative r a n k i n g of subjects is re- markably close in each instance. M E T H O D S A N D D E V I C E S T o o b t a i n an indication of the extent to which each of the methods a n d devices list- ed in the questionnaire, or a d d e d by re- spondents, was employed, the check marks indicating regular use of each of the meth- ods or devices were a d d e d together at the end of each row. Analysis of the d a t a reveals an extremely wide r a n g e in the empl o yment of the various methods a n d devices. U s e r a n g e d from 277, for Conferring with the librarian of the institution, t o o n e f o r Read- ing the faculty library handbook. I n t h e f i r s t instance, most of the respondents indicated that they used regularly the m e t h o d of con- ferring with the librarian in respect to each of the ten subjects of evaluation. With twenty-six to twenty-nine check marks in each of ten columns, the final score of 277 w a s a c h i e v e d . Reading the faculty handbook was suggested by one respondent, who em- ployed it regularly a m o n g other devices when evaluating reader services. Since there was only one check ma rk in this instance the final score was one. It is also desirable to note that some methods a n d devices r a n k e d high in fre- quency because they could be a p p l i e d to the evaluation of a wide r a n g e of subjects, while others were u s e f u l in very limited areas. F o r e x a m p l e , conferences with key members of the library staff might con- ceivably be useful in every area u n d e r study; but talks with deans might have to be limit- ed to only a few subjects. T h e d e a n is not expected to know very m u c h a b o u t the JULY 1958 313 quality of technical services b u t he might be most u s e f u l in discussing the a t t i t u d e of the institution toward the library. Discussion with key staff m e m b e r s scored 185 as a de- vice; with deans, seventy-four. T h e d a t a revealed in the q u e s t i o n n a i r e returns are presented in a series of tables, the first of which, T a b l e IV, is a master list of methods a n d devices a r r a n g e d in the order of the frequency of their use "reg- u l a r l y " by the respondents. T h i s table, how- ever, does not reflect accurately the degree to which each m e t h o d or device is e m p l o y e d a n d could, therefore, be misleading. Addi- tional analysis of t a b u l a t e d responses indi- cated that certain devices were e m p l o y e d more widely than it would at first a p p e a r , as they were used " o c c a s i o n a l l y , " to a large extent. A d d i n g together figures represent- ing " R e g u l a r " a n d " O c c a s i o n a l " use pro- duced another, more significant view of the evaluation techniques. A decision was m a d e finally to present a report of methods a n d devices u n d e r three headings to indicate more precisely the degree to which these de- vices are employed. T h e selection of items to be placed u n d e r each h e a d i n g was based mainly u p o n the q u e s t i o n n a i r e responses a n d partly on the writer's own experience a n d j u d g m e n t . T h e results are offered in the following three tables. It should be emphasized as these tables are e x a m i n e d that they represent only the practice of those who r e s p o n d e d to the ques- tionnaire. It is believed that a representative g r o u p of library evaluators r e s p o n d e d to the q u e s t i o n n a i r e b u t this should not be taken to m e a n that these lists of practices are offered as b e i n g authoritative a n d com- plete. T h e y are, to be sure, highly suggestive, but they d o not represent the intangibles, such as s o u n d experience, g o o d j u d g m e n t , tact, diplomacy, a n d humility, which an evaluator should possess in order to p e r f o r m his work satisfactorily. It should also be emphasized that the practices referred to in these tables are r e p o r t e d as those used dur- ing the course of a visit. CRL statistics a n d A L A s t a n d a r d s may be used with isome fre- quency before a n d a f t e r a visit, a n d j o u r n a l s T A B L E I V M E T H O D S A N D D E V I C E S U S E D D U R I N G E V A L U A T I O N V I S I T S M E T H O D S AND DEVICES F R E Q U E N C Y 1. R e v i e w library sections of i n s t i t u t i o n ' s r e p o r t to t h e M i d d l e States A s s o c i a t i o n . . . 277 2. C o n f e r w i t h chief librarian 277 3. R e a d librarian's a n n u a l reports a n d o t h e r significant reports a n d m e m o r a n d a . . . . 210 4. C o n f e r w i t h c o l l e a g u e s o n e v a l u a t i o n t e a m 194 5. C o n f e r w i t h key library staff m e m b e r s 185 6. C o n f e r w i t h chief librarian u p o n c o n c l u s i o o n of visit 172 6a. R e a d survey reports, if a v a i l a b l e , by o u t s i d e c o n s u l t a n t s to t h e library 157 7. Discuss briefly basic r o u t i n e s a n d p r o b l e m s of m a j o r library d e p a r t m e n t s w i t h their h e a d s 147 8. C o m p a r e library practices w i t h p r e v a i l i n g practices of o t h e r libraries 146 9. E x a m i n e statistics a n d / o r reports of c i r c u l a t i o n 138 10. C o m p a r e library e x p e n d i t u r e s w i t h total i n s t i t u t i o n a l e x p e n d i t u r e s 136 11. E x p l o r e a d m i n i s t r a t i v e structure of library; r e l a t i o n of d e p a r t m e n t a l libraries w i t h m a i n c o l l e g e library or of c o l l e g e a n d s c h o o l libraries w i t h university library . . . . 132 12. C o n f e r w i t h library c o m m i t t e e m e m b e r s 129 13. Inspect book stacks 127 14. I n c o n n e c t i o n w i t h N o . 11 above, c o n f e r w i t h a p p r o p r i a t e d e a n s a n d d e p a r t m e n t h e a d s 123 15. E x a m i n e statistics a n d / o r reports of u s e of n e i g h b o r i n g libraries 121 16. Inspect r e a d i n g r o o m s 117 17. C o n f e r w i t h faculty m e m b e r s o t h e r t h a n library c o m m i t t e e 110 18. Check library h o u r s 107 19. R e a d s a m p l i n g s of m i n u t e s of library c o m m i t t e e s 106 20. E x a m i n e statistics a n d / o r reports of reserves 106 21. Check s e a t i n g capacity 105 22. E x a m i n e statistics a n d / o r reports of interlibrary loans 104 314 COLLEGE AND RESEARCH LIBRARIES 23. E x a m i n e statistics a n d / o r reports of c a t a l o g i n g a n d processing 100 24. R e a d c o l l e g e or university catalogs 98 25. R e a d c o l l e g e or university statutes, if any, r e l a t i n g to library p o l i c y 97 26. I n q u i r e a b o u t r e l a t i o n s h i p s w i t h a n d possible d e p e n d e n c e u p o n n e i g h b o r i n g libraries 96 27. E x a m i n e reports a n d / o r statistics of orders 94 28. E x a m i n e surveys of library resources m a d e by library a n d / o r faculty 89 29. Inspect work r o o m s 87 30. E x a m i n e statistics a n d / o r reports of a u d i o - v i s u a l services 83 31. E x a m i n e library card catalogs 79 32. I n q u i r e a b o u t m e t h o d s a n d q u a l i t y of t e a c h i n g 75 33. C o n f e r w i t h deans 74 34. E x a m i n e statistics a n d / o r reports of u n a v a i l a b l e books 68 35. C o n f e r w i t h p r e s i d e n t 67 36. C o m p a r e library e x p e n d i t u r e s w i t h A C R L statistics in CRL 67 37. C o n f e r w i t h s t u d e n t s at r a n d o m 52 38. R e a d survey reports, if a v a i l a b l e , of t h e w h o l e i n s t i t u t i o n for library i m p l i c a t i o n s 51 39. C o m p a r e library e x p e n d i t u r e s w i t h those of o t h e r libraries in MSA territory 50 40. C o n f e r w i t h a d m i n i s t r a t i v e officers o t h e r than the p r e s i d e n t a n d d e a n s 46 41. Spot-check a v a i l a b i l i t y of books listed in p u b l i c c a t a l o g 39 42. C o n f e r w i t h others in i n s t i t u t i o n u p o n c o n c l u s i o n of visit (others t h a n librarian) 31 43. E x a m i n e desiderata files, if any, in order d e p a r t m e n t 31 44. C o n f e r w i t h r e p r e s e n t a t i v e s t u d e n t s 26 45. C o m p a r e library's e x p e n d i t u r e s w i t h A L A standards (Classification and Pay Plans) 22 46. C o n f e r w i t h h e a d of s t u d e n t b o d y 17 47. Visit classes (other t h a n library) 15 48. C o m p a r e library practices w i t h those of one's o w n library 11 49. Check Reader's Guide to Periodical Literature 9 50. Check Shores, Basic Reference Sources 7 51. C o m p a r e library e x p e n d i t u r e s w i t h R a n d a l l a n d G o o d r i c h , Principles of College Li- brary Administration 6 52. Check L y l e a n d T r u m p e r , Periodicals for the College Libr