kuh.p65 256 College & Research Libraries July 2003 The Role of the Academic Library in Promoting Student Engagement in Learning1 George D. Kuh and Robert M. Gonyea This study examines the nature and value of undergraduate students’ experiences with the academic library. The data represent responses from more than 300,000 students between 1984 and 2002 to the Col­ lege Student Experiences Questionnaire. Although library use did not appear to make independent contributions to desirable outcomes of college, such experiences were related to important educationally valu­ able activities. Because the emphasis a campus places on information literacy is a strong predictor of students becoming information literate, librarians should redouble their collaborative efforts to promote the value of information literacy and help create opportunities for students to evalu­ ate the quality of the information they obtain. t is hard to imagine a college without a library. A required stop on campus tours, the li­ brary is the physical manifes­ tation of the core values and activities of academic life. The size of the collection is used as an indicator of academic quality. Though recent years have not necessar­ ily been kind in terms of budget support, the library’s central role in the academic community is unquestioned. It is almost heretical to ask (given the library’s iconic status as a symbol of aca­ demic values), but just what does the li­ brary contribute to student learning, broadly defined? Student learning cer­ tainly is not the only relevant dimension on which to appraise the library’s value and utility. Nevertheless, in the increas­ ingly harsh light of public accountability and financial constraints, the question has never been more important or timely, nor can it be avoided.2 Three major trends de­ mand an answer. They are (1) unfettered asynchronous access to an exponentially expanding information base; (2) a shift in the focus of colleges and universities from teaching to learning; and (3) the expecta­ tion that all university functions and pro­ grams demonstrate their effectiveness. Awash in Information With unlimited access to information via the Internet, the need for and practical value of a physical repository for printed and other material are less compelling today. On average, college students spend as much time on the Internet as they do studying.3 At the same time, the informa­ tion highway introduces new challenges George D. Kuh and Robert M. Gonyea are affiliated with the College Student Experiences Questionnaire Research Program, Center for Postsecondary Research, Policy, and Planning, School of Education, at Indiana University Bloomington; e-mail: kuh@indiana.edu. 256 mailto:kuh@indiana.edu The Role of the Academic Library in Promoting Student Engagement in Learning 257 for librarians to meet.4 To state the obvi­ ous, not everything available electroni­ cally is valid and reliable. In the past, knowledge gatekeepers (journal editors, publishers) and librarians determined what was worth reading and collecting. Today, students make more of these judg­ ments without assistance. Only about half of all students are confident in their abil­ ity to find good information and about the same percentage admit to having dif­ ficulty in judging the quality and accu­ racy of what they do find.5 For this rea­ son, students must develop a capacity for critical discernment to judge the quality and utility of information, during and after college. The Association of College and Research Libraries (ACRL) refers to the ability to “find, retrieve, analyze, and use information” as “information lit­ eracy.” One cannot become information liter­ ate without first acquiring the founda­ tional skills and competencies tradition­ ally associated with general education— critical thinking and reasoning abilities, written and oral communication skills, and so forth.6 According to Shapiro and Hughes: Information literacy should in fact be conceived more broadly as a new liberal art that extends from know­ ing how to use computers and ac­ cess information to critical reflection on the nature of information itself, its technical infrastructure, and its social, cultural and even philosophi­ cal context and impact—as essential to the mental framework of the edu­ cated information-age citizen as the trivium of basic liberal arts (gram­ mar, logic and rhetoric) was to the educated person in medieval soci­ ety.7 To prepare librarians for the task, ACRL developed five competence stan­ dards and founded an Institute for Infor­ mation Literacy (IIL) that, among other things, assists librarians in working with others in the educational community to promote and cultivate information lit­ eracy. One strategy suggested that librar­ ians move out of the library into class­ rooms where they team-teach courses with faculty colleagues from various dis­ ciplines. Most of this work takes place in lower-division courses where, for better or worse, institutions emphasize general education skills and competencies. At In­ diana University Purdue University In­ dianapolis, for example, a librarian serves on each of the four-person instructional teams (instructor, librarian, academic ad­ visor, student mentor) that deliver the Learning Community course designed for first-year students.8 At Sonoma State University, a librarian teams with the in­ structor of the Freshman Interest Group seminar to increase information compe­ tence.9 Embracing the Learning Paradigm The shift from emphasizing teaching to fo­ cusing on student learning as the primary goal of undergraduate education is gain­ ing traction in all types of postsecondary institutions.10 Accreditors and policy mak­ ers are pushing and applauding this change in emphasis that promises to have profound effects on many aspects of academic life. The implications for the library are plain: Stu­ dents’ experiences with academic libraries should make direct or indirect contributions to desired outcomes of college.11 In addi­ tion to information literacy, are there other outcomes that library experiences could and should foster? The limited evidence on this point is mixed. R. R. Powell summarized evidence that the use of the library correlated with stu­ dent persistence rates and college grades.12 However, he based his conclusions on studies that, for the most part, did not con­ trol for student ability or institutional fac­ tors such as selectivity. A more recent study at Glendale Community College in Cali­ fornia showed that students who partici­ pated in library workshops had much higher pass rates in English and ESL classes, but, again, this study did not ac­ count for student ability.13 Considering fac­ tors that might influence student perfor­ http:ability.13 http:grades.12 http:college.11 http:institutions.10 258 College & Research Libraries July 2003 mance, the relationships between the li­ brary and student performance are less clear. For example, J. C. Ory and L. A. Braskamp reported positive relationships between using the library and gains in critical thinking.14 Others, such as Patrick T. Terenzini and others found negative re­ lationships between library experiences and critical thinking scores.15 The most probable explanation for the contradictory results related to critical thinking and library use is that students use library resources in different ways. To illustrate, library experiences can be di­ vided into two types of activities.16 One is routine, but generally tentative, explo­ ration, such as looking for information, reading assigned reference materials, and using the facility primarily to study. The second type of use—and arguably more powerful in terms of learning—is more focused exploration, analysis, and evalu­ ation of information, driven by learner­ (or collaborative work group) generated questions or, perhaps, stimulated by prob­ lems introduced by the instructor for which library resources are required to solve. Ethelene Whitmire found that the latter type of activity had a significant positive effect on student self-reported critical thinking gains.17 These effects also appeared to be independent of key stu­ dent characteristics such as race and ethnicity.18 Demonstrating the Library’s Educational Value The increasing interest from all quarters in information literacy and student learn­ ing makes it difficult to ignore the hereti­ cal question posed at the outset: To what extent do libraries today contribute to in­ formation literacy and other aspects of student learning? One way to demon­ strate the library’s contribution is to as­ sess whether students’ experiences with the library directly or indirectly contrib­ ute to desired outcomes of college. Using the library also may have salutary effects, such as developing an appreciation of a wide range of literature or different phi­ losophies of life. To obtain and interpret this kind of information, librarians need to understand the conditions that foster learning and how they might indepen­ dently, or with others, assess the out­ comes associated with library experi­ ences. Decades of research on college student development point to two simple propo­ sitions that account for many of the more important influences on student learning. First, the more time and energy students invest in activities related to desired out­ comes of college, the more likely they are to benefit in those areas.19 Second, educa­ tionally effective institutions design ex­ periences that channel students’ energies toward educationally purposeful activi­ ties.20 Unfortunately, relatively little is known about what and how students’ academic library experiences contribute to desired outcomes of college (including information literacy) or about the nature of the relationships between library use and college experiences that research studies show directly affect student learn­ ing, such as student–faculty interaction, writing activities, and so forth. Purpose This study examines the nature and value of students’ experiences with the aca­ demic library. Its aim is to discover the unique contributions of library experi­ ences (including contact with librarians) to the quality of effort students expend in other educationally purposeful activi­ ties, the gains they report making during college, and their overall satisfaction with the college experience. More specifically, the study attempts to answer the follow­ ing questions: 1. Has student use of various library resources changed between 1984 and 2002? That is, given the availability of in­ formation via the Web and other sources, are students using the library more or less for certain reasons (for studying, for find­ ing information)? 2. Is frequent use of the library asso­ ciated with greater gains in information literacy? What does the library contrib­ ute to other desired outcomes of college? http:areas.19 http:ethnicity.18 http:gains.17 http:activities.16 http:scores.15 http:thinking.14 The Role of the Academic Library in Promoting Student Engagement in Learning 259 3. Finally, how does student use of li­ brary resources affect their engagement with effective educational practices? That is, are students who frequent the library more likely to report increased contact with faculty members inside and outside the classroom? Are they more likely to talk with peers about substantive topics such as social, political, and economic issues? Serious conversations with other stu­ dents may be an indicator of the extent to which a college’s general education pro­ gram animates lively discussions beyond the classroom and initiates debates on new topics. Moreover, the more engaged students are in these and other education­ ally purposeful activities, the more likely they are to engage fully in productive ac­ tivities after college, including civic par­ ticipation and so on. Methods Inetrument The College Student Experiences Ques­ tionnaire (CSEQ) assesses the quality of effort students devote to educationally purposeful activities. As mentioned earlier, quality of effort is the single best predic­ tor of what students gain from college; thus, this measure also can be used to esti­ mate the effectiveness of an institution or its component organizations (such as the library) in promoting student learning.21,22 Overall, the CSEQ is considered to have excellent psychometric properties.23 The fourth edition of the CSEQ is made up of 166 items divided into four sec­ tions.24 The first section (18 items) asks for information about the student’s back­ ground (age, year in school, major field, parents’ education), how many hours per week they study, how many hours they work on and off campus, and how they are paying for their education. The sec­ ond section (111 items) contains the 13 College Activities scales (including expe­ riences with the library and computing and information technology) that mea­ sure the amount of time and energy (qual­ ity of effort) students devote to various activities. The fourth edition of the CSEQ contains both a revised library experi­ ences scale and a computing and infor­ mation technology scale that did not ap­ pear on previous editions of the instru­ ment. The response options for these items are: 1 = never, 2 = occasionally, 3 = often, and 4 = very often. This section also includes two questions about the amount of reading and writing students do. The third section (10 items) measures student perceptions of the extent to which their institution’s environment emphasizes im­ portant conditions for learning and per­ sonal development, including the impor­ tance of information literacy. Student re­ sponses are scored on a 7-point scale rang­ ing from 7 (strong emphasis) to 1 (weak emphasis). Three questions gauge student opinions about the quality of relation­ ships with faculty members, administra­ tive personnel, and other students on campus. Two additional questions mea­ sure student satisfaction. In the final sec­ tion, students estimate the extent to which they have gained or made progress since starting college in twenty-five areas that represent desired outcomes of higher education. Response options for the “gains” items are: 1 = very little, 2 = some, 3 = quite a bit, and 4 = very much. Samplee To answer the three guiding research ques­ tions, the authors draw on two overlap­ ping samples of students from the CSEQ Research Program at Indiana University Bloomington. The first sample consists of more than 300,000 students from about 300 different four-year colleges and universi­ ties who completed the second, third, and fourth editions of the CSEQ over a nine- teen-year period (1984 through 2002). The second sample is composed of more than 80,000 full-time students from 131 bacca­ laureate degree-granting institutions who completed the fourth edition of the CSEQ between 1998 and 2002. The background characteristics of the respondents in both samples generally mirror the population of undergraduate students attending four- year colleges and universities with a couple of exceptions. Women and white students are slightly overrepresented; and http:tions.24 http:properties.23 260 College & Research Libraries July 2003 TABLE 1 CSEQ Library Experiences Scale (QELIB) In your experience at this institution during the current school year, about how often have you: Item Name Item Label Response Set LIBI Used the library as a quiet place to read or study materials I = never you brought with you 2 = occasionally LIB2 Found something interesting while browsing in the library 3 = often LIB3 Asked a librarian or staff member for help in finding 4 = very often information on some topic LIB4 Read assigned material other than textbooks in the library (reserve readings, etc.) LIB5 Used an index or database (computer, card catalog, etc.) to find material on some topic LIB6 Developed a bibliography or reference list for a term paper or other report LIB7 Gone back to read a basic reference or document that other authors referred to LIB8 Made a judgment about the quality of information obtained from the library, World Wide Web, or other sources Cronbach's alpha = .80 men, black, and Hispanic students are underrepresented.25 Variables of Interest The particular variables of interest in this study are the eight items that make up the CSEQ library experiences scale (QELIB) (table 1). The scale is reliable (Table 1, Cronbach’s alpha = .80), and the eight items moderately correlate with one another (ranging from .19 to .58; see ap­ pendix A). This study uses three outcome vari­ ables. The first two are composed of out­ comes represented by students’ responses to twenty-five questions about how much progress they have made since starting college (1 = very little, 2 = some, 3 = quite a bit, 4 = very much). The first of these is an Information Literacy Scale (INFOLIT) (table 2), which approximates the skills and competencies ACRL considers impor­ tant for information literacy as reflected by student responses to six “estimate of gains” questions. The second is overall gains, or GAINSUM, the sum of re­ sponses to all twenty-five “estimate of gains” items. (See appendix B for the list of “gains” items.)26 Because the twenty- five “gains” items encompass a holistic set of outcomes in college, GAINSUM is a measure of the student’s perceived over­ all impact of the college experience.27 The last outcome variable is satisfac­ tion (OPINSCOR) and is composed of two CSEQ items: “How well do you like col­ lege?” and “If you could start over again, would you go to the same institution you are now attending?” (See table 3.) Student satisfaction is widely considered an im­ portant indicator of an institution’s com­ mitment to student success, and it is rea­ sonable to expect that library experiences should contribute to this indicator. Addi­ tional statistics for the library scale and the three outcome variables appear in appendix C. Data Analysis To answer the first research question, “Has student use of the library changed over http:experience.27 http:underrepresented.25 The Role of the Academic Library in Promoting Student Engagement in Learning 261 TABLE 2 Information Literacy Scale (INFOLIT) In thinking about your college or university experience up to now, to what extent do you feel you have gained or made progress in the following areas? Item Name Item Label Response Set GNCAREER GNGENLED GNCMPTS GNANALY GNSYNTH GNINQ Gaining a range of information that may be relevant to a career Gaining a broad general education about different fields of knowledge Using computers and other information technologies Thinking analytically and logically Putting ideas together; seeing relationships, similarities, and differences between ideas Learning on your own, pursuing ideas, and finding information you need 1 = Very little 2 = Some 3 = Quite a bit 4 = Very much Cronbach's alpha = .80 time?” the authors examined seven library experience items that remained exactly or essentially the same on the second, third, and fourth editions of the survey, spanning the years 1984 through 2002. One excep­ tion is the second and third edition ques­ tion, “How often have you used a card catalogue.” On the fourth edition of the CSEQ, this item was changed to: “How often have you used an index or database (computer, card catalog, etc.) to find mate­ rial on some topic?” The authors mapped student responses to this set of library ex­ perience items by charting the combined yearly percentage of students responding “often” or “very often” to each item. To answer the second and third ques­ tions, the authors examined the frequen­ cies of responses to the library experi­ ences items by gender, year in school, race, and institutional type. (See appen­ dix D for frequency tables.) The authors also conducted analysis of variance tests to determine whether groups differed sig­ nificantly in their use of the library and in their self-reported gains. Finally, the TABLE 3 Satisfaction with the College EX[erience Scale (OPINSCOR) In thinking about your college or university experience up to now, to what extent do you feel you have gained or made progress in the following areas? Item Name Item Label Response Set LIKECOLL How well do you like college? 1 = I am enthusiastic about it 2 = I like it 3 = I am more or less neutral about it 4 = I don't like it SAMECOLL If you could start over again, would you go to the same institution you are now attending? 1 = Yes, definitely 2 = Probably yes 3 = Probably no 4=No, definitely 262 College & Research Libraries July 2003 TABLE 4 Academic Challenge ltems3! Item Name Item Response Set STUDlES4 Hours per week on out-of-class academic work 1 = Up to 5, 2 = 6-10, 3 = 11-15, 4 = 16-20, 5 = 21- 25, 6 = 26-30, 7 = 30+ READTXT4 READPAK4 WRlTTRM4 Number of texts read Number of course packets read Number of term papers written 1 = none, 2=fewer than 5, 3 = between 5 and 10, 4 = between 10 and 20, 5 = more than 20 COURSE5 COURSE11 COURSE8 FAC9 FAC5 Put together different facts and ideas Worked on project-integrating ideas Applied class material to other areas Worked to meet faculty expectations Worked harder due to instructor feedback 1 = never, 2 = occasionally, 3 = often, and 4 = very often ENVSCH4 ENVCRlT4 Emphasis on developing academic, scholarly, and intellectual qualities Emphasis on developing critical, evaluative, and analytical qualities Cronbach's alpha = .74 7 = strong emphasis to 1 = weak emphasis authors conducted a series of regression analyses to examine the relationships among variables (regression tables are available from the authors). Student characteristics and institutional characteristics can affect student collegiate experiences and outcomes.28 For example, students majoring in the humanities (which include more women than men) may be more likely to use the library facility because the nature of their academic work requires more reading and, therefore, a greater need to obtain a variety of reference material. For this reason, the authors dummy-coded gen­ der (women as reference group) and major field (preprofessional as reference group). They also dummy-coded race and ethnicity (white as reference group) and class level (freshmen as reference group) because the success of these groups of students are of keen interest to institutions and policy mak­ ers. The regression analyses also control for three institutional characteristics: (1) in­ stitutional type as defined by the 2000 Carnegie classification (doctoral/re­ search-extensive universities, doctoral/ research-intensive universities, master ’s colleges and universities, baccalaureate liberal arts colleges, and baccalaureate general colleges), (2) institutional selec­ tivity, and (3) institutional control (pub­ lic and private, with public institutions as reference group).29,30 The Carnegie clas­ sifications were dummy-coded and en­ tered into the models with doctoral/re­ search-extensive universities as the refer­ ence group. Four regression models were con­ structed. In the first model, the Library Experiences scale (QELIB) is the depen­ dent variable and student and institutional characteristics are control variables. Then, selected items were added from the CSEQ College Activities scales that are concep­ tually associated with library use to deter­ mine which ones may account for an ad­ ditional portion of variance in the library scale. These items are use of computer and information technology, course-learning http:outcomes.28 CSEQ Library Items (1983 2002) Freshmen and Sophomores The Role of the Academic Library in Promoting Student Engagement in Learning 263 activities, interactions with faculty mem­ bers, writing experiences, and use of cam­ pus facilities. The three remaining regression models examine the contribution of library experi­ ences to three outcome measures: (1) gains in information literacy (INFOLIT), (2) over­ all gains in college (GAINSUM), and (3) satisfaction with the college experience (OPINSCOR). Control variables in each model include student and institutional characteristics, perceptions of the campus environment, and the academic challenge scale (table 4). The authors controlled for academic challenge because students at in­ stitutions that have high-performance ex­ pectations for academic work are more likely to use the library. Finally, the authors added the library activity items to the model to see if they would explain additional vari­ ance in the outcome measure. In reporting the regression results, the authors will focus only on those findings that are both statistically significant and have reasonable effect sizes. That is, the objective is to identify library experiences that have practical implications as well as statistical significance.32 To do this the au­ thors computed Y-standardized effect sizes by dividing the unstandardized co­ efficient by the standard deviation for the dependent variable.33 They considered ef­ fect sizes greater than |.08| worthy of attention because they represent poten­ tially important relationships between li­ brary experiences, gains from college (in­ cluding information literacy), and student satisfaction.34 Results Tresultsalysis Figures 1 and 2 depict the proportions of first-year and sophomore students (com­ bined) and juniors and seniors (com­ bined) that responded “often” or “very often” to four selected library experiences between 1984 and 2002. These activities are: (1) used the library to read or study, (2) asked a librarian for help, (3) read in the library’s reserve or reference section, and (4) used an index or database. These four experiences showed the greatest changes over the nineteen-year period, with the other four library experiences being generally stable. Because different students and institutions participate in a given year, year-to-year deviations from the trend line are common. Nevertheless, FIGURE 1 CSEQ Library Items (1983-2002) Freshmen and Sophomores - 0 10 20 30 40 50 60 70 1984 1985 1986 1987 1988 1989 1990 1991 199 2 199 3 1994 1995 1996 1 997 1 998 1999 2000 2001 P er ce n t "O ft e n " o r "V e ry O ft e n " us ed library to read or s tudy as ked lib rarian fo r help r ead in res erve or ref s ection us ed index or databas e http:satisfaction.34 http:variable.33 http:significance.32 CSEQ Library Items (1983 2002) Juniors and Seniors 264 College & Research Libraries July 2003 FIGURE 2 CSEQ Library Items (1983-2002) Juniors and Seniors - 0 10 20 30 40 50 60 70 198 4 1 985 1986 1987 1988 1989 19 90 1991 1992 1993 199 4 1 995 1996 1997 1998 1999 20 00 2001 P er ce n t "O ft e n " o r "V e ry O ft e n " us ed li bra ry to r ead or s tudy as ked librarian for h elp read i n r es er ve or ref s ection us ed inde x o r data bas e the overall multiyear trends probably re­ flect meaningful changes over time. Two trends stand out. First, greater numbers of students are using indexes and databases to find information. This likely reflects the rapid and expansive deploy­ ment and use of computers and informa­ tion technology during the past decade that makes more information accessible to more people as well as easier to navigate. To illustrate, in the mid-1980s, only about 30 percent of first-year and sophomore stu­ dents said they frequently used indexes or databases. Beginning in the early 1990s, this percentage jumped to close to half. Juniors and seniors showed similar in­ creases, from about 38 percent in the 1980s to over 60 percent by 2001. The second trend is the decline in the proportion of students who use the li­ brary as a place to read or study. This is probably due to the explosion of the World Wide Web in the mid-1990s, mak­ ing it possible for many students to ac­ cess information and library resources online from their dorm rooms, fraternity and sorority houses, other campus loca­ tions, and off-campus residences.35 An­ other factor may be the availability of ad­ ditional campus venues where students can do academic work, such as computer labs, academic support centers, and study lounges in campus unions or residence halls. These locations may be especially attractive to commuter students if park­ ing near the library is problematic. A less definitive trend is a slight increase in the number of students asking a librar­ ian for help during the 1980s and early 1990s. Librarians about this time began to offer instructional workshops and guidance on how to use the Web.36 Another factor may have been the involvement of librar­ ians in student success programs, such as orientation and first-year student seminars. This behavior varies a bit more from the mid-1990s on, perhaps because librarians were more or less involved in such efforts at the different schools participating in vari­ ous years. What cannot be gleaned from these data is whether the nature of the re­ quests of librarians made by students changed through time. For example, are students more frequently asking librarians for technical assistance with online data­ bases and search engines? Are students ask­ ing for assistance in finding materials con­ tained in the library building? http:residences.35 The Role of the Academic Library in Promoting Student Engagement in Learning 265 Frequency of Library Use Examination of students’ library experi­ ences shows some interesting differences by class, race, major, and institutional type (appendix D). The ANOVA tests support these differences.37 On balance, as stu­ dents move through the college years, they become more information literate each year, a finding corroborated by Steve Jones.38 For example, each successive year from first-year to senior shows a signifi­ cant increase in the frequency of library use. That is, more seniors frequently make judgments about information quality (43%) compared with first-year students (34%); fewer seniors compared with first- year students (18% and 26%, respectively) say they “never” do this. Hispanic, Latino, and Black students use library resources more frequently, whereas White students use libraries the least. Students majoring in humanities and social sciences are, as expected, the most frequent users of the library, as are students who report two or more majors. Students with undecided majors and those majoring in business, math, and science score the lowest on the library scale. Finally, students attending bacca­ laureate liberal arts colleges use the li­ brary more often, whereas those attend­ ing baccalaureate general colleges and doctoral/research-extensive universities do so least often. The next section dis­ cusses whether these differences hold up after controlling for student and institu­ tional variables simultaneously. Regression Results The first regression model uses the library experiences scale (QELIB) as the dependent variable to answer the question, “Who uses the library most?” (See appendix E.) After controlling for student and institutional characteristics, students of color use the li­ brary more frequently compared with white students; students majoring in the humanities and preprofessional fields use the library more often than those majoring in business, math, or science. Access to com­ puting and information technology in­ versely relates to library use and shows a relatively large effect size (.17); that is, stu­ dents who do not have a computer where they live or work (or nearby) tend to use the library more. Perhaps for these students, the library is one place where they can use a computer that, in turn, allows them to ac­ cess databases and obtain information from other libraries. At the institutional level, stu­ dents at doctoral/research-extensive uni­ versities use the library less frequently com­ pared with students attending the other four types of institutions. Academic challenge relates positively to library use. Of the eleven academic challenge items (table 4), five have effect sizes greater than |.08|. These include three items related to course learning ex­ periences (put together different facts and ideas, worked on projects integrating ideas from various sources, and applied class material to other areas in life) and two student–faculty interaction items (worked harder than you thought you could to meet faculty expectations and worked harder due to instructor feed­ back). In addition, all other items in the scale show statistically significant differ­ ences, although with smaller effect sizes. The results from the three regression models predicting desired college out- comes—gains in information literacy, overall gains in college, and satisfaction— appear in appendix F. Taken together, these models indicate that none of the individual library activities appears to have a substantial influence on any of the three outcome variables, after controlling for student and institutional characteris­ tics, perceptions of the environment, and academic challenge. The outcome variable represented in the first regression is information literacy. In this model, transfer students and first- year students make the least progress in information literacy. For first-year stu­ dents, this is surely due to the small amount of time they have been in college. For transfer students, the finding is more difficult to interpret and is cause for con­ cern if this sizeable fraction of students is not gaining as much as other students in this important area. Although students http:Jones.38 http:differences.37 266 College & Research Libraries July 2003 majoring in math and science do not use the library as much as their peers do, they report gaining more in information lit­ eracy relative to preprofessional majors. Humanities majors gain less in informa­ tion literacy (relative to preprofessional majors), after controlling for other factors. Students at doctoral/research extensive universities report the greatest gains in information literacy, followed by students at baccalaureate general colleges, doc­ toral/research-intensive universities, and baccalaureate liberal arts colleges. Finally, as expected, students who perceive that their institution places a strong empha­ sis on acquiring information literacy skills report higher gains in information lit­ eracy. The model predicting overall gains tells a somewhat different story. Women and transfer students report making less progress during college, after controlling for other student and institutional char­ acteristics. Black, Hispanic, and Latino students report greater gains than do white students. In terms of institutional type, students at baccalaureate liberal arts and baccalaureate general colleges report lower gains relative to students in doc­ toral/research-extensive universities. The third model shows that transfer students are less satisfied with their over­ all college experiences, and, as expected, students with higher grades are more sat­ isfied. Black and Asian students are also less satisfied than are white students, but Hispanic students are on par with whites in this category. In general, students at the large doctoral-extensive institutions are more satisfied with college than are stu­ dents at the other four institutional types. It appears that a key to satisfaction may be the quality of relationships with other students. This item shows a large effect size (.21) even after controlling for other factors. Still, as with the previous two, this model produced no significant relation­ ships between library experiences and satisfaction with college, after controlling for student and institutional characteris­ tics, perceptions of the environment, and academic challenge. In summary, frequency of library use varies depending on the type of student and the type of institution. The least fre­ quent library users are White students, math and science majors, those who have ready access to a computer, and those who are attending doctoral-extensive univer­ sities. Those who use the library more fre­ quently report a higher degree of aca­ demic challenge. On balance, library ex­ periences are not directly related to infor­ mation literacy, overall gains in college, or satisfaction with the college experience. Discussion The results of this study indicate that stu­ dent use of the library has changed over time. This is not surprising given the now near-universal access college students have to computing and information tech­ nology. Nonetheless, these data corrobo­ rate anecdotal reports and other studies.39 More important, student contact with li­ brarians has increased somewhat during this period, suggesting that librarians may be becoming more visible and acces­ sible to larger numbers of students. Per­ haps students need help to find good in­ formation and to make judgments about the quality of the information they do find.40 It is supported by the relatively high correlations produced in this study between “asked a librarian” and other be­ haviors such as “used index or database,” “found something interesting while browsing,” and “developed a bibliogra­ phy for a term paper.” At the same time, almost one-fifth of all seniors say they never made judgments about the quality of the information they obtain for use in the academic work. This is an unaccept­ ably high number of students about to graduate from college who, by their own report, are underprepared to live and work in an information-rich world. Smallness begets distinctiveness in American higher education.41 This ap­ pears to be the case for the library as well, as the character of experiences with aca­ demic libraries at small, academically challenging baccalaureate liberal arts col­ leges sets them apart from other types of http:education.41 http:studies.39 The Role of the Academic Library in Promoting Student Engagement in Learning 267 institutions. For example, more students at baccalaureate liberal arts colleges (40%) say they frequently make judgments about the quality of the material than at any other type of institution (33% at doc­ toral/research-extensive universities; 37% at doctoral/research intensive uni­ versities; 34% at master’s institutions, and 33% at baccalaureate general colleges) (appendix E). In addition, library experi­ ences at the baccalaureate liberal arts col­ leges strongly correlated with one another and with other educationally purposeful activities, such as working with a faculty member on research or discussing papers with faculty members. One obvious ex­ planation for this is that because most of these institutions are residential in nature, the library is in close proximity to where students live, making access much easier. In contrast, library use is least frequent at larger doctoral/research-extensive uni­ versities. In part, this may be because of the array of alternate academic support venues such institutions provide, such as computer labs and academic skills cen­ ters. Having these options possibly mutes the impact of the academic library on many of the outcomes measures and re­ duces the necessity that a student must use the library for these vital academic services. In addition, research institutions are more likely to claim better wiring for technology—with broadband access to computer networks, excellent library search engines online, network access in residence hall rooms, and so on. Academic Challenge Matters Size and selectivity are not the only fac­ tors that influence library use. Academic challenge also is important. That is, insti­ tutions that set high standards for aca­ demic work seem to impel students to use a variety of intellectual resources actively, including the library. As a result, students who frequently use library resources are also more likely to work harder than they thought they could to meet a faculty member’s expectations and in response to instructor feedback; and they are as­ signed projects that require integrating ideas, putting different facts and ideas together, and applying class material to other areas in life. In addition, students at academically challenging institutions are more likely to ask a librarian for help, use indexes and databases, and make thoughtful judgments about the quality of information they receive. At the same time, using the library does not appear to be associated with the amount of effort students put forth on their own in many other learning activities, such as the amount of effort they put forth in writing or the frequency with which they con­ verse about substantive matters with peers. The Library's Contribution to Student Success On balance, the results of this study indi­ cate that libraries play an important role in helping the institution achieve its aca­ demic mission. It is particularly gratify­ ing that students of color generally use the library as much or more than do other students, especially Black, Asian and Pa­ cific Islander, Hispanic and Latino stu­ dents. Perhaps students of color find the academic library to be a safe haven, a place that supports and nurtures aca­ demic success in collaboration with peers of the same racial and ethnic background, much in the same way the campus union provides a venue for social gatherings. If so, the library is providing a very valu­ able service for a subset of undergradu­ ates that is increasing in number. The most surprising (and mildly dis­ appointing) finding is that library expe­ riences do not seem to directly contrib­ ute to gains in information literacy, to what students gain overall from college, or to student satisfaction. There are three plausible explanations for this. First, the information literacy scale created from selected CSEQ items may not be a valid proxy; that is, other measures may more accurately estimate information literacy as defined by the ACRL. Second, the lack of baseline measures for information lit­ eracy and the other gains makes it diffi­ cult to draw conclusions from student 268 College & Research Libraries self-reported estimates of their gains.42 For example, students attending different col­ leges or majoring in different fields may start college at various levels of informa­ tion literacy. Some students who report gaining relatively little may have been fairly information literate when they started college. Other students who say they gained a good deal may have started college with a lower level of literacy. So, whereas the latter group may have, in­ deed, gained a substantial amount dur­ ing college, their actual level of informa­ tion literacy may be the same as, or even lower than, their peers who reported making less progress in the area since beginning college. This same caveat holds for the overall gains measure. Finally, as with most other desired outcomes, a va­ riety of experiences during college, inside and outside class, contribute to gains and satisfaction, not just one type of experi­ ence. That is, critical thinking is not pri­ marily or exclusively cultivated in the classroom or in the major field; rather, it is the product of cumulative experiences over time in a variety of venues.43 There is no reason to expect that the relation­ ship between library experiences and gains in information literacy or other ar­ eas would be different. Indeed, the find­ ings of this study offer no silver bullet (or single intervention) that will produce an information-literate college graduate. For example, students who report higher levels of information literacy were attending institutions that emphasized the importance of information literacy and encouraged students to use computers and other information resources. They also performed a good deal of reading, includ­ ing some basic references that are more likely to exist either in the library or online. In addition, students who gained the most in information literacy more often made judgments about the quality of the infor­ mation they obtained. In other words, stu­ dents who make the greatest gains in in­ formation literacy attend institutions that communicate the importance of informa­ tion literacy and practice the skills that lead to information literacy. July 2003 Implicationsf orfPracticefandfAdditional Research This brings us to one of the more impor­ tant findings from this study: Students who perceive that their campus empha­ sizes information literacy gain more in this area, net of other influences. This under­ scores the need to collaborate with class­ room instructors and student affairs pro­ fessionals to deliver clear and consistent messages regarding the value of learning about various sources of information, to require evidence that students make dis­ cerning judgments about the quality of the information they use, and, equally impor­ tant, to give students feedback on the qual­ ity of these judgments. Anecdotal experi­ ence suggests that students are more likely to evaluate critically the quality of sources when instructors explicitly require them to do so.44 This suggests that if institutions are serious about graduating information- literate students, they should require ac­ tivities that give students practice and re­ quire them to demonstrate their competence in evaluating the quality of the information they use. Librarians, faculty members, and others will have to be di­ rectly involved in giving students prompt, ongoing feedback about their perfor­ mance. The University of California at Berkeley is an example of this, where the teaching library and the departments of political science and sociology are design­ ing a graduated program of instruction across the undergraduate years that will require students to use information re­ sources.45 Librarians also might partner with student affairs staff to help them iden­ tify ways to identify students who may be struggling with using information appro­ priately and responsibly. A reasonable amount of interaction with knowledgeable adults on a college cam­ pus is very important to student learning. These interactions are especially valuable when they focus on substantive or course content matters.46 Transfer students are one group of students that would benefit from more attention in this regard. At least 40 percent of seniors attending four-year col­ leges and universities started college at a http:matters.46 http:sources.45 http:venues.43 http:gains.42 The Role of the Academic Library in Promoting Student Engagement in Learning 269 school other than the one from which they are about to graduate.47 Yet, they are gen­ erally less engaged in educationally pur­ poseful activities than are their native stu­ dent counterparts. It is difficult to reach transfer students directly, as they are not concentrated in living units or certain courses. Perhaps librarians could collabo­ rate with academic departments to explore ways to induce transfer students to use the library more frequently and to help them attain levels of information literacy com­ parable to students who start and gradu­ ate from the same college. The California State University system has recognized this need, and its twenty-three campuses are joining with community colleges as well as high schools to improve informa­ tion literacy.48 Higher education needs more investi­ gations into the library’s effectiveness in promoting student learning. One fruitful line of inquiry would be to determine the kinds of student interactions with librar­ ians beyond those represented on the CSEQ that effectively promote learning or affect other aspects of the college ex­ perience. The CSEQ does not ask about whether students made effective use of what they learned in a session focused on information literacy facilitated by a librar­ ian; answers to this and related questions would be very instructive in terms of the library’s contribution. Another needed effort would be to determine which ap­ proaches are most effective in teaching in­ formation literacy. Are these skills and competencies best cultivated through a freestanding course, sprinkled through­ out the curriculum, or learned within the context of the discipline or a specific topic? Other research indicates that stu­ dents learn what they study. It would be useful to compare the information literacy levels of students at institutions that re­ quire library assignments as part of one or more courses with those that do not. Another research question is whether student use of the library and interactions with librarians are associated with per­ sistence and graduation, net of other fac­ tors. Previous research has suggested that library experiences relate positively to persistence and student achievement. Most of this research is dated, however, and did not employ advanced statistical methods that controlled for student abil­ ity or institutional selectivity. A time-honored improvement strategy in higher education and other sectors is to identify high-performing organiza­ tions, find out what they do well, and adapt these promising practices for use in other settings. For example, some in­ stitutions have higher-than-predicted graduation rates and student engagement levels.49 Perhaps students at certain col­ leges and universities use the library more and benefit more than might be predicted, all things being equal. It would be instruc­ tive to learn more about these institutions and their libraries. Limitations This study is limited in that the data are from colleges and universities that volun­ tarily administered the CSEQ. If data from other institutions were included, the find­ ings might change in unknown ways. Another factor that could affect the results is whether additional student-level mea­ sures (e.g., ability, motivation) and insti­ tution-level data (e.g., resources) were in­ cluded in the models. There also is the possibility that, as mentioned earlier, stu­ dents use different baselines when report­ ing gains.50 Despite these limitations, the CSEQ research program represents one of the most extensive national databases with survey information from college students related to their quality of effort and gains from college. It is one of the few multi-in­ stitution sources of information about the undergraduate experience that examines the influence of the library on information literacy and other aspects of student learn­ ing and personal development. Conclusion The results of this exploratory study in­ dicate that library experiences of under­ graduates positively relate to select edu­ cationally purposeful activities, such as using computing and information tech­ http:gains.50 http:levels.49 http:literacy.48 http:graduate.47 270 College & Research Libraries July 2003 nology and interacting with faculty mem­ bers. Those students who more frequently use the library reflect a studious work ethic and engage in academically chal­ lenging tasks that require higher-order thinking. Although certain student back­ ground characteristics (race, major, year in school, transfer status, access to com­ puters) affect the nature and frequency of students’ library activities, the library appears to be a positive learning environ­ ment for all students, especially members of historically underrepresented groups. At the same time, library use does not appear to contribute directly to gains in information literacy and other desirable outcomes. This is not surprising, as rarely does any single experience or set of ac­ tivities during college affect student learn­ ing and personal development one way or the other; rather, what is most impor­ tant to college impact is the nature and breadth of a student’s experiences over an extended period. Academic librarians are well positioned to provide leadership and expertise to out­ comes associated with information literacy. However, higher education institutions should not expect them to do this alone. The findings of this study indicate that it takes a whole campus to produce an in­ formation-literate college graduate. For this reason, librarians would do well to redouble their efforts to collaborate with faculty members, instructional develop­ ment staff, and student affairs profession­ als in promoting the value of information literacy in various in-class and out-of-class activities and to provide students with as many opportunities as possible to evalu­ ate the quality of the information they en­ counter, on and off the campus. Notes 1. This paper was originally prepared for an invited session at the 2003 ACRL National Con­ ference. The authors thank Ann Bristow, Polly D. Boruff-Jones, Ilene Rockham, and Carolyn Walters for their comments and suggestions on an earlier draft of this paper. 2. B. G. Lindauer, “Defining and Measuring the Library’s Impact on Campuswide Outcomes,” College and Research Libraries 59, no. 6 (1998); Measuring Up, Measuring Up 2002: The State-by-State Report Card for Higher Education (San Jose, Calif.: National Center for Public Policy and Higher Education, 2002). 3. Steve Jones, The Internet Goes to College: How Students Are Living in the Future with Today’s Technology (Washington, D.C.: Pew Internet & American Life Project, 2002). 4. K. Dunn, “Assessing Student Information Literacy Skills in the California State Univer­ sity: A Progress Report,” Journal of Academic Librarianship 28, no. 1/2 (2002); I. F. Rockman and Gordon W. Smith, “A Multi-dimensional Project to Assess Student Information Competence Skills,” paper presented at the E-Learn Conference, Montreal, 2002. 5. National Center for Postsecondary Improvement, “The Landscape: A Report to Stake­ holders on the Condition and Effectiveness of Postsecondary Education,” Change 33, no. 3 (2001); Outsell, Managing Online Information to Maximize Corporate Intranet ROI (2001) [cited July 2001]). Available online from http://w.moreover.com/. 6. Lindauer, “Defining and Measuring the Library’s Impact on Campuswide Outcomes.” 7. J. J. Shapiro and S. K. Hughes, “Information Literacy as a Liberal Art: Enlightenment Proposals for a New Curriculum,” Educom Review 31, no. 2 (1996): 2. 8. P. Boruff-Jones, personal communication, November 2002. 9. K. Brodsky and S. Toczyski, “Information Competence in the Freshman Interest Group at Sonoma State University,” paper presented at the First Year Experience Conference, Orlando, 2002. 10. Robert B. Barr, and John Tagg, “From Teaching to Learning: A New Paradigm for Under­ graduate Education,” Change 27 (Nov. /Dec. 1995); John Tagg, The Learning Paradigm College (Bolton, Mass.: Anker, 2003). 11. Lindauer, “Defining and Measuring the Library’s Impact on Campuswide Outcomes”; R. A. Wolff, “Rethinking Library Self-studies and Accreditation Visits,” in The Challenge and Practice of Academic Accreditation: A Sourcebook for Library Administration, ed. E. D. Garten (Westport, Conn.: Greenwood, 1994). 12. R. R. Powell, “Impact Assessment of University Libraries,” Library and Information Science Research 14 (1992). 13. Glendale Community College, “Information Competency Improves Grades” (2001). 14. J. C. Ory and L. A. Braskamp, “Involvement and Growth of Students in Three Academic http:http://w.moreover.com The Role of the Academic Library in Promoting Student Engagement in Learning 271 Programs,” Research in Higher Education 28 (1988). 15. Patrick T. Terenzini, “Influences Affecting the Development of Students’ Critical Think­ ing Skills,” Research in Higher Education 36, no. 1 (1995); Patrick T. Terenzini, et al., “First-genera­ tion College Students: Characteristics, Experiences, and Cognitive Development,” Research in Higher Education 37, no. 1 (1996). 16. C. Robert Pace, Measuring the Quality of College Student Experiences. An Account of the De­ velopment and Use of the College Student Experiences Questionnaire (Los Angeles: Higher Education Research Institute, 1984). 17. Ethelene Whitmire, “Development of Critical Thinking Skills: An Analysis of Academic Library Experiences and Other Measures,” College and University Research Libraries 59, no. 3 (1998). 18. ———, “Racial Differences in the Academic Library Experiences of Undergraduates,” Journal of Academic Librarianship 25, no. 1 (1999). 19. Alexander W. Astin, “Student Involvement: A Developmental Theory for Higher Education,” Journal of College Student Personnel 25 (1984); Ernest T. Pascarella and Patrick T. Terenzini, How College Affects Students, The Jossey-Bass Higher and Adult Education Series (San Francisco: Jossey-Bass, 1991). 20. George D. Kuh, J. H. Schuh, E. J. Whitt, and associates, Involving Colleges: Successful Ap­ proaches to Fostering Student Learning and Development outside the Classroom (San Francisco: Jossey Bass, 1991); Roy Romer, Making Quality Count in Undergraduate Education (Denver, Colo.: Educa­ tion Commission of the States, 1995). 21. Pace, Measuring the Quality of College Student Experiences. 22. George D. Kuh, “Assessing What Really Matters to Student Learning: Inside the National Survey of Student Engagement,” Change 33, no. 3 (2001). 23. Peter T. Ewell, and Dennis P. Jones, Indicators of “Good Practice” in Undergraduate Educa­ tion: A Handbook for Development and Implementation (Boulder, Colo.: National Center for Higher Education Management Systems, 1996); Robert M. Gonyea, Kelly Kish, George D. Kuh, Richard Muthiah, and Auden Thomas CSEQ: Norms for the Fourth Edition (Bloomington, Ind.: Indiana University Center for Postsecondary Research, Policy, and Planning, 2003). 24. Robert C. Pace, and George D. Kuh, College Student Experiences Questionnaire (fourth edi­ tion) (Bloomington, Ind.: Center for Postsecondary Research and Planning, 1998). 25. The respondents in the first sample who completed the CSEQ between 1984 and 2002 include 60 percent women; 80 percent are white, 6 percent black, 3 percent Hispanic, 6 percent Asian, and 4 percent other race or ethnicity. First-year students total 35 percent, sophomores 21 percent, juniors 17 percent, and seniors 26 percent. Of the respondents in the second sample who completed the fourth edition of the CSEQ, 61 percent were women and 77 percent were white, 8 percent Asian or Pacific Islander, 5 percent black, 3 percent Mexican-American, Puerto Rican, or other Hispanic, 1 percent American In­ dian, 3 percent multiracial, and 3 percent other race or ethnic identity. Approximately 43 percent were first-year students, 20 percent sophomores, 17 percent juniors, and 20 percent seniors. About 20 percent were majoring in a preprofessional program (e.g., agriculture, education, communications, and health- related fields); 11 percent in social sciences (e.g., multidisciplinary studies, sociology, and public admin­ istration); 16 percent in mathematics, science, or related area (e.g., computer science and engineering); 8 percent in the humanities (e.g., ethnic studies, foreign languages, history, and visual and performing arts); and 15 percent in business. Four percent were undecided as to major field, and 21 percent had two or more majors. In terms of institutional type, 38 percent were from twenty-nine doctoral/research­ extensive universities, 13 percent from seventeen doctoral/research-intensive universities, 33 percent from forty-one masters’ colleges and universities, 8 percent from twenty-one baccalaureate liberal col­ leges, and 9 percent from twenty-three baccalaureate general colleges; Carnegie Foundation for the Advancement of Teaching, Carnegie Classification of Institutions of Higher Education, 2000 edition (Menlo Park, Calif.: Carnegie Foundation for the Advancement of Teaching, 2000); Robert M. Gonyea, Kelly Kish, George D. Kuh, Richard Muthiah, and Auden Thomas CSEQ: Norms for the Fourth Edition (Bloomington, Ind.: Indiana University Center for Postsecondary Research, Policy, and Plan­ ning, 2003). 26. George D. Kuh, Nick Vesper, Mark R. Connolly, and C. Robert Pace, “College Student Experiences Questionnaire: Revised Norms for the Third Edition,” (Bloomington, Ind.: Center for Postsecondary Research and Planning, Indiana University, 1997). 27. Note that INFOLIT is a subset of GAINSUM. 28. Pascarella and Terenzini, How College Affects Students. 29. Carnegie Foundation for the Advancement of Teaching, Carnegie Classification of Institu­ tions of Higher Education. 30. Barron’s Profiles of American Colleges, 23rd ed. (Hauppauge, N.Y.: Barron’s Educational Se­ ries, 1998). 31. Response values on items appended by a ‘4’ were mathematically collapsed to four-point range, giving all items an equal portion of the total scale score. 32. Jacob Cohen, Statistical Power Analysis for the Behavioral Sciences, 2nd ed. (Hillsdale, N.J.: L. 272 College & Research Libraries July 2003 Erlbaum Associates, 1988). 33. R. Light and D. Pillemer, “Numbers and Narrative: Combining Their Strengths in Re­ search Reviews,” Harvard Educational Review (1982). 34. The expression “|.08|” should be read “the absolute value of .08.” 35. I. Rockman, personal communication, December 19, 2002. 36. Ibid. 37. ANOVA results are not reported in this paper but are available from the authors. 38. Jones, The Internet Goes to College. 39. Ibid. 40. Dunn, “Assessing Student Information Literacy Skills in the California State University”; Rockman and Smith, “A Multi-dimensional Project to Assess Student Information Competence Skills.” 41. B. Clark, “The Organizational Saga in Higher Education,” in ASHE Reader in Organization and Governance in Higher Education, ed. R. Birnbaum (Washington, D.C.: Association for the Study of Higher Education, 1972); George D. Kuh and Elizabeth J. Whitt, The Invisible Tapestry: Culture in American Colleges and Universities, ed. J. D. Fife ASHE-ERIC Higher Education Report No. 1 (Wash­ ington, D.C.: Association for the Student of Higher Education, 1988); B. K. Townsend, L. J. Newell, and M. D. Wiese, Creating Distinctiveness: Lessons from Uncommon Colleges and Universities, AAHE­ ERIC/Higher Education Report, no. 6 (Washington, D.C.: The George Washington University, School of Education and Human Development, 1992). 42. Ernest Pascarella, “Using Student Self-reported Gains to Estimate College Impact: A Cau­ tionary Tale,” Journal of College Student Development 42 (2001). 43. Pascarella and Terenzini, How College Affects Students. 44. Carolyn Walters, personal communication, December 22, 2002. 45. P. D. Maughan, Information Literacy Survey (Berkeley: University of California, Berkeley Library, 2002). 46. George D. Kuh, and Shouping Hu, “The Effects of Student–Faculty Interaction in the 1990s,” Review of Higher Education 24, no. 3 (2001). 47. George D. Kuh, “What We’re Learning about Student Engagement from NSSE,” Change 35, no. 2 (2003). 48. Dunn, “Assessing Student Information Literacy Skills in the California State University.” 49. Kuh, “What We’re Learning about Student Engagement from NSSE.” 50. Pascarella, “Using Student Self-reported Gains to Estimate College Impact.” References Astin, Alexander W. 1984. “Student Involvement: A Developmental Theory for Higher Educa­ tion.” Journal of College Student Personnel 25: 297–308. Barr, Robert B., and John Tagg. 1995. “From Teaching to Learning: A New Paradigm for Under­ graduate Education.” Change 27 (November/December): 13–25. Barron’s profiles of American colleges. 23rd ed. 1998. Hauppauge, N.Y.: Barron’s Educational Series. Brodsky, K., and S. Toczyski. 2002. “Information Competence in the Freshman Interest Group at Sonoma State University.” Paper presented at the First Year Experience Conference, Orlando, Fla. Carnegie Foundation for the Advancement of Teaching. 2000. Carnegie Classification of Institu­ tions of Higher Education, 2000 edition. Menlo Park, Calif.: Carnegie Foundation for the Ad­ vancement of Teaching. Clark, B. 1972. “The Organizational Saga in Higher Education.” In ASHE Reader in Organization and Governance in Higher Education, ed. E. Birnbaum. Washington, D.C.: Association for the Study of Higher Education. Cohen, Jacob. 1988. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, N.J.: L. Erlbaum Associates. Dunn, K. 2002. “Assessing Student Information Literacy Skills in the California State University: A Progress Report.” Journal of Academic Librarianship 28 (1/2): 26–35. Ewell, Peter T., and Dennis P. Jones. 1996. Indicators of “Good Practice” in Undergraduate Education: A Handbook for Development and Implementation. Boulder, Colo.: National Center for Higher Education Management Systems. Glendale Community College. 2001. “Information Competency Improves Grades.” Gonyea, Robert M., Kelly Kish, George D. Kuh, Richard Muthiah, and Auden Thomas. 2003. CSEQ: Norms for the Fourth Edition. Bloomington, Ind.: Indiana University Center for Postsecondary Research, Policy, and Planning. Jones, Steve. 2002. The Internet Goes to College: How Students Are Living in the Future with Today’s Technology. Washington, D.C.: Pew Internet & American Life Project. The Role of the Academic Library in Promoting Student Engagement in Learning 273 Kuh, G. D. 2003. “What We’re Learning about Student Engagement from NSSE.” Change 35(2). Kuh, George D. 2001. Assessing What Really Matters to Student Learning: Inside the National Survey of Student Engagement.” Change 33(3): 10–17, 66. Kuh, George D., and Shouping Hu. 2001. “The Effects of Student–Faculty Interaction in the 1990s.” Review of Higher Education 24(3): 309–32. Kuh, George D., J. H. Schuh, E. J. Whitt, and Associates. 1991. Involving Colleges: Successful Approaches to Fostering Student Learning and Development outside the Classroom. San Francisco: Jossey Bass. Kuh, George D., Nick Vesper, Mark R. Connolly, and C. Robert Pace. 1997. College Student Experi­ ences Questionnaire: Revised Norms for the Third Edition. Bloomington, Ind.: Center for Postsecondary Research and Planning, Indiana University. Kuh, George D., and Elizabeth J. Whitt. 1988. The Invisible Tapestry: Culture in American Colleges and Universities, ed. J. D. Fife. ASHE-ERIC Higher Education Report No 1. Washington, D.C.: Association for the Student of Higher Education. Light, R., and D. Pillemer. 1982. “Numbers and Narrative: Combining Their Strengths in Re­ search Reviews.” Harvard Educational Review (1982): 1–26. Lindauer, B.G. 1998. “Defining and Measuring the Library’s Impact on Campuswide Outcomes.” College and Research Libraries 59(6): 546–63. Maughan, P. D. 2002. Information Literacy Survey. Berkeley: University of California, Berkeley Li­ brary. Measuring Up. 2002. Measuring Up 2002: The State-by-State Report Card for Higher Education. San Jose, Calif.: National Center for Public Policy and Higher Education. National Center for Postsecondary Improvement. 2001. “The Landscape: A Report to Stakehold­ ers on the Condition and Effectiveness of Postsecondary Education.” Change 33(3): 27–42. Ory, J. C., and L. A. Braskamp. 1988. “Involvement and Growth of Students in Three Academic Programs.” Research in Higher Education 28: 116–29. Outsell. 2001. Managing Online Information to Maximize Corporate Intranet ROI [cited July 2001]. Available online from http://w.moreover.com/. Pace, C. Robert. 1984. Measuring the Quality of College Student Experiences. An Account of the Devel­ opment and Use of the College Student Experiences Questionnaire. Los Angeles: Higher Education Research Institute. Pace, C. Robert, and George D. Kuh. 1998. College Student Experiences Questionnaire (fourth edi­ tion). Bloomington, Ind: Center for Postsecondary Research and Planning. Pascarella, Ernest. 2001. Using Student Self-reported Gains to Estimate College Impact: A Cau­ tionary Tale.” Journal of College Student Development 42: 488–92. Pascarella, Ernest T., and Patrick T. Terenzini. 1991. How College Affects Students, The Jossey-Bass Higher and Adult Education Series. San Francisco: Jossey-Bass. Powell, R. R. 1992. “Impact Assessment of University Libraries.” Library and Information Science Research 14: 254. Rockman, I. F., and Gordon W. Smith. 2002. “A Multi-dimensional Project to Assess Student Information Competence Skills.” Paper presented at the E-Learn Conference, Montreal. Romer, Roy. 1995. Making Quality Count in Undergraduate Education. Denver, Colo.: Education Commission of the States. Shapiro, J. J., and S. K. Hughes. 1996. “Information Literacy as a Liberal Art: Enlightenment Proposals for a New Curriculum.” Educom Review 31(2): 31–35. Tagg, John. 2003. The Learning Paradigm College. Bolton, Mass.: Anker. Terenzini, Patrick T. 1995. “Influences Affecting the Development of Students’ Critical Thinking Skills.” Research in Higher Education 36(1): 23–39. Terenzini, Patrick T., et al. 1996. “First-generation College Students: Characteristics, Experiences, and Cognitive Development.” Research in Higher Education 37(1): 1–22. Townsend, B. K., L. J. Newell, and M. D. Wiese. 1992. Creating Distinctiveness: Lessons from Un­ common Colleges and Universities, AAHE-ERIC/Higher Education Report, No. 6. Washington, D.C.: The George Washington University, School of Education and Human Development. Whitmire, Ethelene. 1998. “Development of Critical Thinking Skills: An Analysis of Academic Library Experiences and Other Measures.” College and University Research Libraries 59(3): 266– 73. ———. 1999. “Racial Differences in the Academic Library Experiences of Undergraduates.” Jour­ nal of Academic Librarianship 25(1): 33–37. Wolff, R. A. 1994. “Rethinking Library Self-studies and Accreditation Visits.” In The Challenge and Practice of Academic Accreditation: A Sourcebook for Library Administration, ed. E. D. Garten. Westport, Conn.: Greenwood. http:http://w.moreover.com 274 College & Research Libraries July 2003 APPENDIX A CSEQ Library Scale Inter-Item Correlation Coefficient" LIB1 LIB2 LIB3 LIB4 LIB5 LIB6 LIB7 LIB8 LIB1 Used the library to study LIB2 Found something interesting browsing .37 LIB3 Asked a librarian! staff member for help .19 .31 LIB4 Read assigned material, not texts .40 .39 .30 LIB5 Used index or database to find material .26 .39 .38 .38 LIB6 Wrote bibliography for a term paper .21 .27 .33 .32 .58 LIB7 Gone back to read basic reference .26 .37 .28 .36 .36 .43 LIB8 Made a judgment about quality .21 .31 .23 .27 .42 .44 .39 of information *Item-total correlations range from .40 to .62, indicating that each item contributes substan­ tially to the scale. http:quality.21 http:reference.26 http:paper.21 http:material.26 http:texts.40 http:browsing.37 The Role of the Academic Library in Promoting Student Engagement in Learning 275 APPENDIX B CSEQ Gain Scales and Items. Category Item Name Item Label General Education GNARTS GNLIT GNHIST GNWORLD GNPHILS GNGENLED Understanding and enjoyment of art, music, drama Acquaintance with, and enjoyment of, literature Knowledge of history Knowledge about different parts of the world and people Awareness of different philosophies, cultures, ways of life Broad general education Personal Development GNVALUES GNSELF GNOTHERS GNTEAM GNHEALTH Values and ethical standards Self-understanding Ability to get along with others Teamwork skills Good health habits and physical fitness Science and Technology GNSCI GNTECH GNCONSQ Science and experimentation Science and technology developments Consequences of science and technology Vocational Preparation GNVOC GNSPEC GNCAREER Job or work skills Background for further education Career information Intellectual Development GNWRITE GNSPEAK GNCOMPUT GNANALY GNQUANT GNSYNTH GNINQ GNADAPT Writing Presenting and speaking Computers and other information technologies Analytical and logical thinking Quantitative problem solving Synthesis ability Self-directed learning Adapting to change Response set for all Gains items: 1 = Very little, 2 = Some, 3 = Quite a bit, 4 = Very much *GAINSUM Cronbach’s alpha = .92; item-total correlations range from .39 to .68 APPENDIX C Descriptive Statistics for Dependent Variables Used in the Study Measure Valid N Missing N % Missing Mean S.E.M. Std. Dev. Skewness Kurtosis QELIB 78425 1844 2% 17.0 0.02 4.6 0.37 0.08 INFOLIT 76987 3282 4% 17.7 0.01 3.5 -0.29 -0.25 GAINSUM 75103 5166 7% 67.8 0.05 13.1 -0.07 -0.14 OPINSCOR 78487 1782 2% 6.3 0.01 1.5 -0.76 0.19 All four scales have a minimal percentage of missing values and good normal curve properties as indicated by skewness and kurtosis values in the normal range (between -1 and +1). 276 College & Research Libraries July 2003 APPENDIX D Frequencies to Library Experience Items by Sex, Class, Race, and Institutional Type Frequency of Responses to CSEQ Library Experiences Items by Sex Male Female Response Options Col% Col% Used the library to study Never 24.6 23.3 Occasionally 46.2 48.7 Often 17.5 16.8 Very often 11.6 11.2 Found something interesting browsing Never 33.5 36.2 Occasionally 45.7 47.2 Often 14.7 11.6 Very often 6.2 4.9 Asked a librarian/staff member for help Never 29.9 23.0 Occasionally 49.7 52.6 Often 15.6 18.0 Very often 4.8 6.4 Read assigned material, not texts Never 32.5 31.2 Occasionally 45.1 44.3 Often 16.4 17.4 Very often 6.0 7.1 Used index or database to find material Never 13.3 9.1 Occasionally 39.9 34.1 Often 30.0 33.0 Very often 16.9 23.8 Wrote bibliography for a term paper Never 20.9 16.8 Occasionally 40.4 34.2 Often 25.8 28.6 Very often 13.0 20.5 Gone back to read basic reference Never 53.9 58.3 Occasionally 34.0 30.6 Often 8.7 7.6 Very often 3.4 3.6 Made a judgment about quality of Never 24.3 22.6 information Occasionally 39.5 41.0 Often 23.7 24.0 Very often 12.6 12.4 The Role of the Academic Library in Promoting Student Engagement in Learning 277 APPENDIX D Frequencies to Library Experience Items by Sex, Class, Race, and Institutional Type Frequency of Responses to CSEQ Library Experiences Items by Class First-year Sophomore Junior Response Options Col% Col% Col% Used the library to Never 25.5 21.6 22.6 study Occasionally 47.8 47.6 47.0 Often 16.5 17.9 17.7 Very often 10.1 12.9 12.7 Senior Col% 23.3 48.5 16.9 11.3 Found something Never 39.2 34.7 32.1 interesting browsing Occasionally 44.9 46.8 48.6 Often 11.6 13.1 13.3 Very often 4.3 5.4 6.0 29.6 48.4 14.7 7.3 Asked a librarian/staff Never 28.4 26.1 25.0 member for help Occasionally 48.4 52.4 53.0 Often 17.4 16.1 16.4 Very often 5.8 5.4 5.7 19.9 55.9 18.0 6.2 Read assigned material, Never 37.4 29.2 28.8 not texts Occasionally 42.0 46.3 45.0 Often 15.1 17.3 18.0 Very often 5.5 7.1 8.2 24.3 48.3 19.9 7.5 Used index or database Never 12.9 10.2 9.9 to find material Occasionally 37.1 38.7 35.6 Often 30.8 31.9 32.2 Very often 19.2 19.2 22.3 7.3 33.0 33.4 26.3 Wrote bibliography for Never 20.9 18.4 17.0 a term paper Occasionally 35.7 39.8 37.3 Often 26.8 27.0 28.2 Very often 16.5 14.8 17.5 13.9 34.7 28.6 22.8 Gone back to read Never 61.1 58.1 54.0 basic reference Occasionally 29.0 31.8 33.8 Often 7.2 7.4 8.1 Very often 2.8 2.8 4.1 47.5 36.5 10.5 5.5 Made a judgment about Never 25.7 23.7 21.7 quality of information Occasionally 40.2 41.5 40.3 Often 22.8 23.4 24.5 Very often 11.3 11.4 13.5 18.7 39.9 26.2 15.3 278 College & Research Libraries July 2003 APPENDIX D Frequencies to Library Experience Items by Sex, Class, Race, and Institutional Type Frequency of Responses to CSEQ Library Experiences Items by Race and Ethnicity Response Options Col% Col% Col% Col% Col% Mexican- American, Puerto Asian, Black, Rican or Pacific African White, Other Islander American Caucasian Hispanic Other Race Used the library Never 14.7 21.4 25.3 18.6 22.3 to study Occasionally 45.5 47.2 48.3 46.9 46.9 Often 21.6 18.2 16.3 19.3 17.8 Very often 18.2 13.2 10.2 15.2 13.1 Found something Never 30.1 27.4 36.9 29.1 30.9 interesting Occasionally 48.7 46.6 46.5 44.8 46.7 browsing Often 14.3 19.1 11.8 17.9 15.3 Very often 6.9 7.0 4.9 8.2 7.1 Asked a librarian! Never 27.4 17.6 26.1 25.1 24.7 staff member for Occasionally 53.3 48.1 51.9 47.5 49.5 help Often 14.7 23.7 16.7 19.2 18.1 Very often 4.6 10.7 5.3 8.2 7.6 Read assigned Never 28.7 28.7 32.4 29.0 30.7 material, not texts Occasionally 46.2 42.9 44.9 42.4 43.0 Often 17.0 19.5 16.6 20.1 17.7 Very often 8.1 9.0 6.1 8.5 8.6 Used index or Never 11.7 10.2 10.6 10.5 10.9 database to find Occasionally 37.0 32.6 36.8 34.9 34.2 material Often 30.8 31.3 32.0 31.0 31.3 Very often 20.5 25.9 20.6 23.6 23.6 Wrote Never 20.2 18.5 18.0 18.6 19.4 bibliography Occasionally 39.9 33.5 36.6 35.2 35.3 for a term paper Often 24.6 27.5 27.8 27.4 27.3 Very often 15.3 20.5 17.6 18.8 18.1 Gone back to read Never 50.9 49.1 58.3 49.3 54.0 basic reference Occasionally 35.4 35.0 31.1 34.9 32.2 Often 9.8 10.9 7.4 10.3 9.3 Very often 3.9 5.0 3.2 5.5 4.4 Made a judgment Never 25.4 26.8 22.8 22.5 23.1 about quality Occasionally 39.3 37.3 41.3 37.3 37.1 of information Often 23.7 22.7 23.8 25.8 24.2 Very often 11.6 13.1 12.1 14.4 15.6 The Role of the Academic Library in Promoting Student Engagement in Learning 279 APPENDIX D Frequencies to Library Experience Items by Sex, Class, Race, and Institutional Type Frequency of Responses to CSEQ Library Experiences Items by Carnegie Classification' Response Options Col% Col% Col% Col% Col% Liberal Doctoral- Doctoral- Arts General Extensive Intensive Master's Colleges Colleges Used the library Never 22.9 24.1 25.5 14.8 28.9 to study Occasionally 47.1 47.4 48.9 46.3 48.1 Often 17.3 17.0 16.4 20.8 15.1 Very often 12.7 11.5 9.1 18.1 7.9 Found something Never 38.0 32.3 35.0 23.7 38.4 interesting browsing Occasionally 45.5 42.8 48.4 50.4 46.6 Often 11.4 16.0 12.5 17.4 11.1 Very often 5.1 8.9 4.1 8.6 3.8 Asked a librarian/staff Never 29.5 23.5 23.0 23.0 24.6 member for help Occasionally 51.4 48.2 51.8 56.2 51.1 Often 14.5 19.5 19.1 16.1 18.3 Very often 4.6 8.8 6.1 4.7 6.1 Read assigned Never 33.3 31.7 32.5 17.0 35.1 material not texts Occasionally 43.9 43.7 45.9 44.5 44.7 Often 16.1 17.4 16.5 24.6 15.4 Very often 6.7 7.2 5.2 13.9 4.8 Used index or database Never 11.6 10.2 10.4 5.6 14.0 to find material Occasionally 37.6 34.1 36.2 30.4 40.5 Often 30.8 33.2 32.7 33.8 28.8 Very often 20.1 22.5 20.8 30.2 16.7 Wrote bibliography Never 20.7 19.3 17.3 12.0 16.3 for a term paper Occasionally 37.9 35.7 35.9 33.8 37.5 Often 25.2 27.6 29.4 29.5 27.7 Very often 16.1 17.4 17.4 24.8 18.4 Gone back to read Never 58.1 55.9 58.1 44.5 56.2 basic reference Occasionally 30.9 31.7 30.9 39.0 33.2 Often 7.6 8.6 7.8 10.7 7.4 Very often 3.4 3.8 3.1 5.8 3.2 Made a judgment Never 24.1 24.0 23.3 16.4 24.3 about quality of Occasionally 39.0 39.7 42.3 38.9 41.4 information Often 23.8 23.9 23.4 27.1 23.4 Very often 13.1 12.4 11.0 17.6 10.9 *Carnegie Foundation for the Advancement of Teaching, Carnegie Classification of Institutions of Higher Education. 280 College & Research Libraries July 2003 APPENDIX E Variables with Significant and Reasonable Effects on the Library EX[eriences Scale Independent Variables Effect Size' Student Race and ethnicity Black, African American 0.17 Characteristics (white as reference Asian, Pacific-Islander 0.15 group) Hispanic or Latino 0.16 Other race or ethnicity 0.09 Major Categories Math and science -0.12 (Preprofessional as Humanities 0.0S reference group) Social sciences Business -0.09 Undecided Two or more majors Year in school Sophomore 0.0S (First-year students as Junior 0.14 reference group) Senior Transfer status (1 = transfer, 0 = non-transfer) -0.09 Access to a computer (1 = yes, 2 = no) 0.17 Expect to enroll for an advanced degree (1 = yes, 2 = no) 0.11 Carnegie classification Doctoral-intensive 0.24 (Doctoral-extensive as Master's 0.1SInstitutional reference group) Liberal arts colleges 0.21Characteristics General colleges 0.10 STUDIES Hours out-of-class academic workAcademic READTEXT Number of texts readChallenge READPAK Number of course packets readScale Items WRITTRM Number of term papers written COURSE5 Put together different facts and 0.0S ideas COURSE11 Worked on project integrating 0.19 ideas from various sources COURSES Applied class material to other 0.0S areas in life FAC9 Worked harder than thought to 0.10 meet faculty expectations FAC5 Worked harder due to 0.11 instructor feedback ENVSCH Environmental emphasis on scholarly, academic, and intellectual qualities ENVCRIT Environmental emphasis on developing critical, evaluative,N = 69,923; R2 = .25 and analytical qualities *Y-standardized effect size (unstandardized B coefficient divided by the standard deviation of the dependent variable). http:other0.0S http:integrating0.19 http:colleges0.21 http:asMaster's0.1S http:classificationDoctoral-intensive0.24 http:non-transfer)-0.09 http:asJunior0.14 http:schoolSophomore0.0S http:Business-0.09 http:asHumanities0.0S http:science-0.12 The Role of the Academic Library in Promoting Student Engagement in Learning 281 APPENDIX F Predictors of Three Outcome Variables from the CSEQ" Independent Variables Dependent Variables Satisfaction Information Overall with College Literacy Gains Score Experience Category Variable Effect Size Effect Size Effect Size Student Age Characteristics Sex (0 = male, 1 = female) -.08 Env. emphasis: Info. literacy .13 Transfer Status Grades at this college Expect to enroll for an advanced degree First-generation student -.09 -.08 -.09 .09 Race and ethnicity Black, African American Asian, Pacific-Islander Hispanic or Latino Other race or ethnicity .11 .12 -.27 -.29 -.08 Major category Math and science Class standing Sophomore Humanities Social sciences Business Undecided Multiple majors Junior Senior .16 -.09 .21 .30 .34 .15 .25 .33 .39 -.15 -.11 Institutional Barron's selectivity code Characteristics Control (0 = public, 1 = Institution Type Doctoral-Intensive Perceptions of Env. emphasis: Aesthetics Environment Env. emphasis: Diversity private) Master's I and II Baccalaureate Liberal Arts Baccalaureate General -.13 -.09 -.15 -.11 -.10 -.10 -.14 -.20 -.23 -.32 skills Env. emphasis: Vocational Env. emphasis: Practical courses Relationships: Other .21 students http:literacy.13 http:female)-.08 282 College & Research Libraries July 2003 APPENDIX F (CONTINUED) Predictors of Three Outcome Variables from the CSEQ" Independent Variables Dependent Variables Satisfaction with College Experience Effect Size Category Variable Information Literacy Effect Size Overall Gains Score Effect Size Perceptions of Environment Relationships: Administra- tive personnel Relationships: Faculty members Academic Challenge CSEQ Academic Challenge Scale � Library Experiences Used the library to study Found something interesting browsing Asked a librarian/staff member for help Read assigned material, not texts Used index or database to find material Wrote bibliography for a term paper Gone back to read basic reference Made a judgment about quality of information .31Model R2 .39 .44 *Only Y-standardized effect sizes greater than |.08| are shown.