College and Research Libraries Managing Change: Supporting Users of Automated Systems Gary Marchionini and Danuta A. Nitecki Rapidly evolving technology is bringing change to academic libraries in unprecedented ways. One strategy for managing the changes due to online integrated library systems is to provide user and staff training. A review of the training literature and the results of three projects for supporting patron use of this system are presented. Results indicate that academic library pa- trons have little trouble using such systems and respond favorably to all modes of training. The results suggest that training should focus on generic strategies for information seeking rather than on the functional use of integrated library systems. entral to all American libraries is providing access to informa- tion, a mission typically met through the acquisition and systematic organization of published ma- terials. The organization is reflected in or- derly tools, such as catalogs, which offer users means to identify what is locally owned. With the development of elec- tronic integrated library systems this bib- liographic control often becomes a power- ful intermediary for connecting the users' needs and the library's holdings through multiple access points. Moreover, elec- tronic systems have the potential to ex- tend users' information-seeking capabili- ties beyond local holdings. The introduction of new technologies and techniques for information processing is a double-edged sword. It promises to im- prove efficiency and effectiveness in han- dling information for librarians and pa- trons alike, but it also brings the stresses of change due to new patterns of behavior, equipment failure, and new ways of thinking. Managing the change related to automation has emerged as a central con- cern of librarians today. A natural and log- ical response to this concern is to provide training and support services for staff and patrons. This article describes three re- search efforts related to an a~tomated li- brary system and discusses their implica- tions for patron support services in academic libraries. LIBRARY INSTRUCTION Most American academic libraries ac- cept the obligation of providing instruc- tion as part of their mission. A distinction is often made between user orientation to local resources and services and biblio- graphic instruction for library research methods. Both types of instruction must include introduction to and use of online tools through which a user can meet spe- cific information needs. As machine-readable library databases become available through campuswide networks connecting powerful work sta- tions, the library's traditional instruc- tional role may be challenged and cer- tainly will be changed. Computer center staff also see patron assistanc~ as a natural extension of their mission. Debates and cooperative efforts are emerging as librari- ans and computer center personnel try to resolve their traditional functions in light of new technological developments. 1 It is essential that libraries provide effec- Gary Marchionini is Assistant Professor, College of Library and Infonnation Seroices, and Dan uta A. Nitecki is Associate Director for Public Seroices at the University · of Maryland, College Park, Maryland 20742. 1M tive instruction and assistance at the com- fort level as well as the competence level. To provide such instruction, methods and materials that are customized for adult learners in an academic setting must be developed and integrated with existing methods and materials. Everyone agrees that support services specific to electronic information systems be provided; what is not clear is how best to structure, deliver, and evaluate these services. Population and Content Fo~emo~t in any instructional plan is consideration of the client population. In academic libraries, the clients are adults who are generally intelligent and knowl- edgeable about information services. Re- · search in adult learning indicates that adults vary greatly in capabilities, are self- ~irected learners, have large and formal- IZed. knowledge bases, carefully consider the mvestment of time in learning, and learn from a variety of media. 2 These results should guide the design of training procedures and materials for users of aca- demic libraries. Short, intensive units that are individualized and delivered in a vari- ety of formats are best for these learners. Since materials of this nature can be ex- pensive to develop and maintain, it is es- sential that the amount and type of con- tent be carefully considered. Attention to ~ormation-seeking strategies and judg- mg relevancy should be the ultimate goals of most user training, not the mechanics of using a particular index or system. As the studies described below demonstrate, at- tention to the mechanics of using auto- mated systems in the context of an aca- demic library does not optimize library resources or users' learning time. Instruc- tions. for usin~ a system can be effectively and mexpens1vely provided through the provision of easily accessible, self-directed materials. Various media are available for this purpose and considerable research has been conducted on the role of media in delivering instruction. Instructional Media Researchers in many disciplines have conducted investigations comparing the Managing Change 105 effectiveness of various media for instruc- tion. Research summaries of media selec- tio!). report no conclusive evidence for the effectiveness of various media. 3 R. Clark even suggests that the medium makes no difference in learning. 4 Robert Reiser and Robert Gagne present an overview of models used for selecting media for in- structional purposes in which they sug- gest that the specific instructional vari- ables included in the model contribute a bias for the eventual determination of me- dia. 5 Considering the number and com- plexity of the variables involved, it is not surprising that conclusive evidence has not been found. There have been many studies of in- structi~mal media particular to library in- st~uct.wn. M~st have involved library skills mstruchon, and there is little doubt that this is effective and useful for stu- dents. 6 Comparative examinations of dif- ferent media have also been conducted. Frank Kuo compared six modes of instruc- tion for library skills and concluded that simply adding visual support to instruc- tion did not affect learning, and an audio- visual presentation supplemented by a li- brarian was superior to self-directed audiovisual instruction. 7 Since time on task was not controlled, these results are not surprising. Julia ~aldwin and Robert Rudolph com- pared shde/tape to guided tour instruction with no conclusive results. 8 Timothy Jewell compared self-paced workbook in- struction to media-assisted lectures and found that students generally favored workbooks. 9 Other researchers found use o.f programmed instruction books supe- nor to the lecture method in teaching bib- liographic skills. 10 Susan Rawlins found no statistically sig- nificant differences between computer- assisted instruction (CAl) and lecture but students reacted more favorably to CAL 11 M. Guilfoyle noted that student assistants felt CAl was a good way to learn library skills. 12 Maria Sugranes and James Neal evaluated a se1f-instruction course with fa- vorable results but made no formal com- parisons:13 After reviewing various in- structional media, Nal'lcy Fjallbrant and Ian Malley conclude, "In practice a combi- 106 College & Research Libraries nation of teaching methods and media can be expected to provide the best basis for programmes of library user education, dif- ferent methods being adapted to different parts of the programme and to the teach- ers and students concerned. ''14 Overall, there is no conclusive evidence from psychology, education, or library sci- ence that one mode of instruction is supe- rior to another in providing instruction, al- though attitudes toward nontraditional methods are reported to be superior. Taken as a whole, the evidence suggests that differences in learning are more likely due to interactions among individual characteristics, subject matter, and media. A pragmatic approach would be to pro- vide a variety of media and allow the learner to choose which to use. THE STUDIES In an environment such as the campus of the University of Maryland at College Park, the responsibility to provide train- ing in using the online integrated library system, including the public query feature of the circulation system and the online catalog, is a natural extension of the li- brary's traditional role of offering both bibliographic instruction and user orienta- tion. During the 1984-85 academic year three studies of the university's integrated library system were conducted. In one, patron instruction was the primary inter- est, in another, patron use was examined, and in the third, staff support service for patrons was considered. Learning to Use the Online Circulation System A comparative experiment was con- ducted in spring 1985 to determine which of three modes of instruction was superior in introducing library users to a newly im- plemented online circulation system and to relate each of these modes to individual characteristics. The research was spon- sored by the Council on Library Resources and the full report is available as an ERIC document. 15 A brief summary of proce- dures and results follows. An instructional sequence or script was written, which presented an overview of the integrated library system and gave de- tailed instruction for using the online cir- March 1987 culation subsystem. · The functions tar- geted for instruction were searching for an item by author, title, or call number; plac- ing a hold; and obtaining patron informa- tion. Examples were selected to illustrate both concepts and procedures and sample screen displays were prepared. The script was evaluated for accuracy and clarity by 1· public services staff in the campus li- braries and revised as necessary. The script was used to develop three parallel modules, each using a distinct mode of instruction. The three modes were printed text, videotape, and interac- tive (hands-on). The text version was sim- ply the original script with example screen displays embedded at appropriate places. The interactive version used the same text but subjects actually worked through the examples at a terminal by using the text in a ''cookbook'' fashion. The video version consisted of a narration of the script with actual screen displays shown as examples. An achievement test was designed to measure procedural (what to do), inter- pretative (what does a screen mean), and general (what is included in the database) knowledge about the system. A demographic section was appended to the final instrument. A total of fifty-one subjects, mostly graduate library science students, were assigned to one of the three treatment groups. Each subject had twenty minutes to use the instructional unit and ten minutes to complete the achievement test. Individual characteris- tics considered in the study were gender, previous computer experience (five-point scale), previous online circulation experi- ence with the system studied (two-point scale), previous online circulation experi- ence with other systems (two-point scale), age, and native language. In general, subjects were able to master the use of the online circulation subsystem regardless of the mode of instruction. The overall mean score on the achievement test was 87 percent. A one-way analysis of variance across treatment groups resulted in no statistically significant differences (F = 2.81, p > .05) among the three in- structional modes. A weak relationship (Spearman r = .22, p > .05) was found between treatment and willingness to use the system in the future. Subjects in the interactive group were most likely to use the system in the future, with the video- tape group least likely to use it in the fu- ture. No statistically significant correla- tions between individual characteristics and achievement were found . The results of this study suggest that pa- tron instructional needs for using an on- line circulation system can be met with a variety of short, introductory modules that are freely available for individual pa- tron use. Searching the Online Public Access Catalog Also sponsored by the Council on Li- brary Resources, this study explored user search behavior when using an online public access catalog (OP AC) .16 One of the purposes of the study was to develop an experience base for the design and evalua- tion of OP AC training methods and mate- rials. Both search patterns and search results were examined with respect to in- dividual characteristics of subjects. Search patterns were also related to the system's user interface. Thirty-nine subjects, mostly graduate li- brary science students (but not any who participated in the previous study), were given thirty minutes to conduct two researcher-assigned searches using the OP AC. One search was simple and fo- cused and the other was complex and open-ended. No instruction in using the system was provided and users had no previous experience with the OP AC. All user keystrokes and system responses were captured in the system's log file. A relevancy score based upon the research team's judgments was computed for each search. Subject searches that exhibited a variety of access methods (e.g. subject, author, keyword, etc.) were classified as heterogeneous searches and those that contained a single access method were termed homogeneous. Together with questionnaire responses upon completion of the searches, these user/system interac- tions served as raw data for examination of search patterns and search results. About two-thirds of the subjects used homogeneous search patterns, and no sta- tistically significant (at .05 level) correla- tions were found between individual Managing Change 107 characteristics and search pattern. Sub- jects were generally successful on the cri- terion measures of number of hits, rele- vancy score, and satisfaction with results. No statistically significant (.05) correla- tions between individual characteristics and success measures were found. Over- all, subjects found the system to be easy to use and very seldom used the help func- tion. Although some suggestions were made for improving the command set and screen displays and making command summaries available near the stations, subjects found the system to be easy to use even with no instruction. It should be noted that Boolean search capabilities were not yet implemented at the time the study was conducted. Based on the results, substantial re- source investments in training materials seems imprudent. However, command summaries should be available at the ter- minals and short, introductory modules in a variety of media formats could be made available for independent patron use. User Assistance Survey Another method for managing the in- troduction of an integrated library system is to assign library staff to areas where ter- minals are placed so introductions can be given and questions answered. Because this can be expensive and can divert staff from other functions, a pilot interview/ survey was conducted in the fall 1985 se- mester to determine whether patrons used or required personnel dedicated to the online system. An interview protocol was developed and used with randomly selected users to assess their satisfaction, success, and per- ceptions about ease of use. After using the system, subjects were also given an op- portunity to have a graduate library sci- ence student reconduct their search for them. Of the sixteen users interviewed, two-thirds were successful in finding in- formation they sought even though half of the respondents had never used the sys- tem before. All users said that the system was easy to use, and only one user asked to have the search augmented by the inter- viewer. Only one-third of the respondents who used the OP AC looked in the card catalog, 108 College & Research Libraries serials list, or other sources in addition to the OP A C. This is significant because at the time of the study the OPAC did not contain all items in the card catalog. Signs pointing this out were prominently dis- played. This study suggests that full-time staff need not be assigned to the terminal area. It does reinforce the need to help pa- trons look beyond the iritegrated library system for comprehensive information and suggests that library instruction focus on the many sources of information avail- . able in academic libraries rather than on the mechanics of using a particular sys- tem. CONCLUSIONS AND RECOMMENDATIONS It is clear from these studies that users in an academic library are ready and able to use integrated library systems with little formal training with respect to the me- chanics of using electronic equipment and systems. What is needed is library instruc- tion that stresses the various sources of data available both locally and remotely; strategies for searching electronic sys- tems, e. g., broadening and narrowing tac- tics, using synonyms, and differentiating between full-text, numeric, and biblio- graphic databases; and ways of judging relevancy. The particulars of using a sys- tem, e.g., what keys to press for an author search or what masking or truncation key to use, can easily be provided with flip charts or posters available near the work stations. A second recommendation is that in- structional modules that address topics users need help with should be provided in short, intensive units and in a variety of media that allow self-directed study. It is not the medium that makes a difference, but the instructional content. Providing a variety of media may help attract a wider range of users to the system. It must be noted that systems are be- March 1987 coming easier for uninitiated patrons to use, and although this suggests the need for less staff assistance in how to use the system, it does not diminish the need for staff assistance in when to use the system and what the results of using the system mean. The ease of getting some response-regardless of how incomplete it may be-seems to generate a false credi- bility that t~e system's response is ade- quate or even comprehensive. This chal- lenges librarians to raise users' consciousness, to stir curiosity in the in- completeness of our controls over the wealth of information available, and to raise self-confidence in their need to know. These are the real challenges of change that must be met by academic li- braries. One thing is certain, change will con- tinue. We are in the midst of a transition period as users increase their interaction with machine-readable databases. Since these databases are not yet fully devel- oped and new ones are emerging, most li- braries are required to make simultaneous use of electronic and manual systems. Search languages are not standardized, let alone natural language-like, and are sure to change as pseudointelligent front ends and gateways evolve. In the midst of these changes, the focus of the academic librari~n should be to pro- vide settings and tones that induce com- fort and a sense of human control over systems; guidance on what resources most effectively meet various information retrieval needs; basic instruction on search techniques; and assurance that the entire process is not difficult and is evolv- ing toward more efficient, effective, and easy-to-use systems. Such demands may not allow for the most thorough examina- tion of every nuance of use of online sys- tems, but should be met with greater flexi- bility and attitude conditioning to cope with and welcome inevitable change. REFERENCES 1. Patricia Battin, ''The Library: Center of the Restructured University,'' College and Research Libraries 45:170-76 (May 1984); Raymond Neff, "Merging Libraries and Computer Centers: Manifest Des- tiny or Manifestly Deranged?," EDUCOM Bulletin 20:8-12 (Winter 1985). Managing Change 109 2. James Kidd, How Adults Learn (New York: Association Press, 1973); Alan Knox, Adult Development and Learning (San Francisco: Jossey-Bass Publishers, 1977). 3. W. Schramm, Big Media Little Media (Beverly Hills, Calif.: Sage Publications, 1977). 4. R. Clark, "Reconsidering Research on Learning from Media," Review of Educational Research 53:445-59 (Winter 1983). 5. Robert Reiser and Robert Gagne, "Characteristics of Media Selection Models," Review of Educa- tional Research 52:499-512 (Winter 1982). 6. Larry Hardesty, Nicholas Lovrich and James Mannon, "Evaluating Library-Use Instruction," Col- lege and Research Libraries 40:309-17 (July 1979); Laslo Nagy and Martha Lou Thomas, ''An Evalua- tion of the Teaching Effectiveness of Two Library Instructional Videotapes,'' College and Research Libraries 42:26-30 (Jan. 1981); Richard Wood, "The Impact of a Library Research Course on Stu- dents at Slippery Rock University," Journal of Academic Librarianship 10:278-84 (Nov. 1984). 7. Frank Kuo, "A Comparison of Six Versions of Science Library Instruction," College and Research Libraries 37:287-90 (July 1973). 8. Julia Baldwin and Robert Rudolph, "The Comparative Effectiveness of a Slide/Tape Show and a Library Tour," College and Research Libraries 40:31-35 (Jan. 1979). 9. Timothy Jewel, "Student Reactions to a Self-Paced Library Skills Workbook Program: Survey Evi- dence," College and Research Libraries 43:371-78 (Sept. 1982). 10. Patricia Kenney and Judith McArthur, "Designing and Evaluating a Programmed Library Instruc- tion Text," College and Research Libraries 45:35-42 (Jan. 1984); Thomas Surprenant, "Learning The- ory, Lecture, and Programmed Instruction Text: An Experiment in Bibliographic Instruction," College and Research Libraries 43:31-37 (Jan. 1982). 11. Susan Rawlins, "Technology and the Personal Touch: Computer-Assisted Instruction for Library Student Workers," Journal of Academic Librarianship 8:26-29 (Mar. 1982). 12. M. Guilfoyle, "Comptuer-Assisted Training for Student Library Assistants," Journal of Academic Librarianship 10:333-36 (Jan. 1985). 13. Maria Sugranes and James Neal, "Evaluation of a Self-Paced Bibliographic Instruction Course," College and Research Libraries 44:444-57 (Nov. 1983). 14. Nancy Fjallbrant and Ian Malley, User Education in Libraries (London: Clive Bingley, 1984), p.67. 15. Gary Marchionini and Dan uta Nitecki, Learning to Use an Online Circulation System (Final Report to the Council on Library Resources, ERIC Document Reproduction Services, ED 257 471, 1985). 16. See Gary Marchionini and Dean Gattone, Searching the Online Public Access Catalog (Final Report to the Council On Library Resources, ERIC Document Reproduction Services, ED 258 585, 1985).