452 Impact of Undergraduate Students’ Library Use on Their Learning beyond GPA: Mixed-Methods Approach Jung Mi Scoulas and Sandra L. De Groote* The research aims to identify the primary reason for students’ library use, to explore how they characterize academic success, and to further examine the impact of library usage on student learning outcomes beyond GPA, through a mixed-methods ap- proach. By surveying and interviewing 27 undergraduate students during 2019–2020, the findings revealed that almost every participant strongly believed that their use of library space was positively associated with their GPA, independent of how often they would use the library. While many participants defined academic success as getting a good grade, some identified academic success as developing application skills and obtaining new knowledge. Introduction University of Illinois Chicago (UIC) library conducted a biannual online survey for undergradu- ate and graduate students in 2016 and 2018 to better understand users’ needs and examine how the academic library contributes to students’ academic success. One of the key findings from the 2018 survey was the negative correlation between student library visits and their GPA, although students’ use of library resources (such as journal articles and databases) was positively associated with their GPA.1 This finding is contrary to the earlier studies indicat- ing that students’ library visits were positively associated with students’ academic success.2 However, given that correlation does not guarantee causation, it is not easy to clearly explain this negative relationship. As such, the current study aims to identify the primary reasons for students’ library use (in person and online) and to examine whether academic-related activi- ties such as studying in the library, using library resources, and attending library instruction have an impact on students’ learning beyond GPA, through a mixed-methods approach. A unique contribution of the study is a design comparing groups of students in three ranges of GPA. Additionally, the authors aim to further explore how students perceive their experience in the library and how students perceive their own learning through a qualitative approach. The findings from the current research project provide qualitative evidence about how students * Jung Mi Scoulas is Assistant Professor and Assessment Coordinator in the University Library at the University of Illinois Chicago; email: jscoul2@uic.edu. Sandra L. De Groote is Professor and Head of Assessment and Scholarly Communications in the University Library at the University of Illinois Chicago; email: sgroote@uic.edu. ©2022 Jung Mi Scoulas and Sandra L. De Groote, Attribution-NonCommercial (https://creativecommons.org/licenses/ by-nc/4.0/) CC BY-NC. mailto:jscoul2@uic.edu mailto:sgroote@uic.edu https://creativecommons.org/licenses/by-nc/4.0/ https://creativecommons.org/licenses/by-nc/4.0/ Impact of Undergraduate Students’ Library Use on their Learning beyond GPA 453 see their library use (library space, website, and instruction) affecting their learning beyond what is measured through students’ GPAs. Literature Review Academic libraries’ endeavors to demonstrate the library’s value for students’ academic suc- cess and learning have yielded fruitful literature in recent years. Many academic libraries used students’ Grade Point Average (GPA) as an outcome variable to show how libraries contributed to student academic success and their learning, which generated a relevant and consistent find- ing: students’ library use (such as library instruction, database use, and checkouts) is positively associated with their GPA.3 Part of the research exploring these relationships was spearheaded through the efforts of the Association of College and Research Libraries’ (ACRL) Value of Aca- demic Libraries Initiative4 and ACRL’s whitepaper: Academic Library Impact: Improving Practice and Essential Areas to Research.5 One of the six priority areas outlined in the whitepaper focused on student learning and success, which provided guidance on “Quantify the library impact on student success.” In more recent years, the literature pertaining to academic libraries’ contribu- tions to academic success (GPA) are steadily increasing such as Ula Gaha, Suzanne Hinnefeld, and Catherine Pellegrino’s study examining the correlations between library instruction and GPA;6 Tiffany LeMaistre, Qingmin Shi, and Sandip Thanki’s investigation of the relationship between library online resource use and GPA;7 and Francesca Marineo and Qingmin Shi’s examination of the relationships between library use sessions and GPA.8 Given that GPA has been widely used in various fields as a quantifiable indicator of student academic success and learning (for instance, in Psychology, Educational Psychol- ogy, and Library and Information Sciences), it is very important to have empirical evidence demonstrating that academic libraries have a positive impact on students’ academic success using GPA. However, the notion of academic success can vary subjectively by individual, and academic success is influenced by various factors.9 In a recent meta-analysis study, M. Brooke Robertshaw and Andrew Asher found many studies where the statistical analysis and interpretations were not compelling, and libraries contribution to students’ academic success in fact had very little impact.10 Additionally, in the trends section of the 2018 ACRL Academic Library Trends and Statistics Survey, the findings revealed that 45 percent of respondents from doctoral institutions did not have any outcome metrics to track students’ success.11 Only 11 percent tracked and assessed “student retention rate,” followed by 9 percent that used GPA as a student outcome. The vast majority of respondents from doctoral institutions (83%) reported that they do not know the correlations between students’ library use and their retention and graduation rate. In an effort to help academic libraries that do not have the expertise to measure library value or want to compare the impact of their library programs with other libraries, the ACRL launched Project Outcome for Academic Libraries in April 2019.12 This is a free online toolkit providing standardized surveys for assessing and analyzing learning outcomes so academic libraries can apply their results to make changes for improvement or use the data for bench- marking with other libraries across the nation. In the survey, there are four key outcomes that measure the library programs and services (such as library instruction, space, digital, and special collections) in the areas of knowledge, confidence, application, and awareness. As of April 2020, 476 academic and research libraries have created surveys, and more than 40,000 responses were collected.13 454 College & Research Libraries May 2022 Despite this effort, there are still gaps and issues in the practices of assessing the value of academic libraries. A quantitative approach to measure the relationships between library use and student academic success may not fully capture the impact of the library on students’ success and learning. To address this issue, in a recent study, Jennifer Mayer, Rachel Dineen, Angela Rockwell, and Jayne Blodgett stated that they were mindful of the concerns reflected by nonpersuasive findings from previous studies due to minimal statistical impact raised by other researchers.14 So they designed and conducted a research project using both the quan- titative and qualitative approach: to examine the impact of library resources and services on student success (persistence and GPA) in the aggregate level through a quantitative approach and to explore how students define academic success and their perceptions of the library’s role at the individual level through a qualitative approach. The findings from the quantita- tive data indicated that students’ use of resources and services, with the exception of research consultations, were statistically and positively associated with their persistence. Students’ use of resources and services were found to have a minimal positive effect on students’ GPA. The findings from qualitative data revealed that students perceived success as “made their best possible effort,” “gained knowledge or learned something new regardless of their assigned grade,” and “applying knowledge to real-life.”15 The qualitative findings further uncovered that students perceived the library as “a place to focus, study, to meet friends, to get away and to get things done,” suggesting that “library provides an important physical environment to enable them to achieve their fullest potential in academic ability.” The current study aims to measure the library values on the student’s academic success by adopting the Project outcome model to examine the impact of library usage on student learning beyond GPA (quantitative approach) and by exploring how they characterize academic success (qualitative approach). Methodology The purpose of this research is to examine students’ primary reasons for using the library and to measure whether students’ library use has an impact on students’ outcomes (GPA, knowledge, competence, awareness, and belonging). In addition, this research explores how students perceive their use of the library, either in person or online, impacting their learning. To achieve these goals, the current study employs a mixed-methods approach: the quantita- tive method (survey) and the qualitative method (interview)—more specifically, the embed- ded design. The embedded design is a mixed-methods design using both quantitative and qualitative data; one type of dataset supports and plays a supplementary role in a research study based primarily on the other dataset.16 In this research project, the embedded design with a one-phase approach is used; quantitative data (survey) as a secondary data type is embedded within a qualitative design (interviews). The authors collected quantitative data as a supplementary data and primarily used qualitative data to understand in depth how students’ library experiences have an impact on their academic success through their narra- tives. Given that this research project is more weighted in the qualitative methods design, the study’s qualitative approach does not make a claim of generalizability, but rather seeks deeper understanding of the complexity of student perspectives. This research study was approved by the Institutional Review Board (IRB) at the Institution (Research protocol # 2019-0538). Measures Three instruments were used in this research project: a recruitment survey, a pre-interview Impact of Undergraduate Students’ Library Use on their Learning beyond GPA 455 survey, and interview protocols. All of the instruments were developed by the authors adapting a locally developed student survey by the University Library17 and Project Outcome model.18 All of the instruments were reviewed by the University Library’s Assessment Coordinator Advisory Committee to ensure the questions measured as intended and to enhance the clar- ity of the questions. • Recruitment Survey: The recruitment survey consisted of six questions: screening ques- tions to determine if students were eligible to participate in the survey, questions related to frequency of library in-person visits and online library usage, the student’s GPA, and contact information for follow-up for participating in the interviews (see appendix A). Online library usage was defined as students accessing library resources and services through the library website. • Pre-interview Survey: The pre-interview survey consisted of 22 questions including the following: primary reasons for using the university library, frequency of using the physi- cal and online library, desired learning outcomes (knowledge, confidence, application, attitude) related to library space use, library instruction participation, library website use, students’ attitude toward the library, and demographic questions such as ethnicity, age, and gender (see appendix B). Learning outcomes for space, website, and library instruction were adapted from the Project Outcome Model.19 • Interview Protocol: The interview protocol consisted of 10 interview questions including the following: student’s overall experience and perceptions on library use, interaction with library staff, participation and perceptions of library instruction, challenges using the library, areas for library improvement, and the impact of library use on the student’s academic success (see appendix C). Data Collection Recruitment procedures. Recruitment flyers were distributed at the UIC Library and the Honors College. Flyers were also distributed through social media (Facebook, Twitter, and Instagram) and library news and were displayed in the Richard J. Daley Library building (June 12, 2019~March 23, 2020). Students who were interested in participating in an interview were asked to complete the recruitment survey (see appendix A). Any undergraduate students who reported never using the physical library and never using the online library were directed to exit the survey. Once a demographic was filled (for example, students with a high GPA who only used the library online), additional subjects from the demographic were not contacted for interviews. Screening procedures. The authors used purposive sampling, determining which ele- ments should be included in the sample.20 Based on the respondents’ criteria, the frequency of library use, and their GPA, the authors carefully selected the eligible participants includ- ing balancing the representation from racial/ethnic groups. Students were selected based on their reported GPA and frequency of library use (in person and online) and were contacted individually asking if they were still interested in participating. Given the specific criteria required in the sample, the process of recruiting procedures as well as the interviews were conducted between July 2019 and February 2020. During this period, a total of 170 students completed a recruitment survey. Among them, 39 cases were deleted because of blank re- sponses in either eligibility questions or contact information. A total of 27 participants were selected based on the criteria of each category (low, medium, and high GPA) and frequency 456 College & Research Libraries May 2022 of library use (in person and online). There was representation of students in each category. Once all participants were recruited, the recruitment information was deleted. Interview procedures and sample size. Interviews were conducted by the first author in the university library. The in-person interviews began by completing a print survey (see appendix B) for 5–10 minutes, followed by an interview that spanned less than an hour (see appendix C). Prior to each interview, participants were asked to sign an informed consent form. At the end of the interview, students were asked to verify their GPA by logging into their student account on the university’s system. At the completion of the interview, participants received $25 for their participation. As the interviews went on, the authors determined the sample size for this mixed-method study by using two factors: external factor and data saturation. In the process of recruiting the participants, COVID-19 occurred, and it remained uncertain when interviewing students in person would be able to resume. The external factor is a potential threat that alters the con- sistency of the qualitative research (interview format and students’ library experience before and during COVID). To follow the evidence-based guidance, the authors used data saturation. Saturation is widely acceptable in qualitative research for determining sample size. Guided by the definition of saturation, “the point in coding when you find that no new codes occur in the data”21 and “additional data do not lead to any new emergent themes,”22 the first author discovered the repetitive responses at the interviews after 20 interviews and when reviewing and coding procedures, both authors learned that no new codes emerged and recurrence of the similar responses after the 23rd interview. External factor (COVID) and reaching a point where there are no further new codes and themes emerging were critical factors at which the authors decided to stop interviewing and analyze the data that had already been collected. As such, a total of 27 participants were selected for this mixed-methods study. This sample size is within the scope of published qualitative studies. Data Analysis Quantitative data from survey questions were stored directly in the web-based online survey, Qualtrics. Descriptive statistics were directly analyzed in Qualtrics: frequency, mean, and crosstabs. Qualitative data from interviews were audiorecorded during the interviews and transcribed by a transcribing company. All 27 transcripts were imported into ATLAS.ti (ver- sion 9) and were analyzed using content analysis to better understand students’ motivations of using the library and examine their perspectives of the impact of library on their academic success. Content analysis was extensively used for qualitative research.23 More specifically, summative content analysis was used when “keywords are derived from interest of researchers or review of literature.”24 The first author initiated the coding and went over the coding with the second author. After agreement with the coding, the authors split half of the transcripts and coded the interviews separately. The codes were reviewed by the authors as a whole and were merged. This procedure was repeated until the authors had agreement on the final codes and themes. Finally, quantitative data was exported into Excel, and their library experiences and the impact of their library use on their learning were simultaneously examined with their interview responses. Limitations Students were self-selected to participate in interviews. Thus, it is possible that their perspec- http://ATLAS.ti Impact of Undergraduate Students’ Library Use on their Learning beyond GPA 457 tives differ from students who did not choose to engage in conversation with a librarian. For example, it is possible that participants’ perspectives on the library were more positive than students overall, or that they were more confident than students on average. The study’s qualitative approach does not make a claim of generalizability; rather, it seeks deeper un- derstanding of the complexity of student perspectives. In particular, comparing students in different GPA ranges is a unique contribution. Results Quantitative Findings As shown in table 1, more than 66 percent of respondents were either sophomore (33.3%) or junior (33.3%), followed by freshman (22.2%) and senior (11.1%). All the students were be- tween the ages of 18 and 25 years. In all, 70 percent of the respondents were female. About 60 percent of respondents were from Liberal Arts and Sciences, and the remaining respondents were from Business Administration (11.1%), Education, Engineering, and Nursing (7.4% each), and Applied Health Science and Architecture, Design and Arts (3.7% each). Fully 40 percent were Asian/Pacific Islander, followed by Black or African American (22.2%), White (18.5%), Hispanic or Latino (14.8%), and International (3.7%). The vast majority of students owned a computer at home (92.3%) and had an in- ternet access at home (95.2%). Nearly three quarters (72%) of respondents had their own study space at home. More than four out of five (81%) respondents lived off campus, with most of these students living less than 10 miles from the university campus (81%). Students’ Primary Reasons for Using the Library. Students were asked their pri- mary reasons for coming to the library. The top three reasons students reported were: studying alone (24.7%), working on assign- ments (19.8%) that required using computers or printers to complete the assignments, or preparing for exams (17.3%). More than half of students responded that the top reason for using the online library was for “accessing library e-books and journals etc. to help me complete assignments” (57.7%). For those who reported coming to the physical library, about half of students chose a large, enclosed study room on the first floor (51.9%). The remaining students indicated they selected space on other floors (floors 2 to 4) within the library. Students were further asked what specific space they use the most. About 60 percent of students reported using TABLE 1 Students’ Demographic Information (n=27) n % Gender Female 19 70.4% Male 8 29.6% Class Status Freshman 6 22.2% Sophomore 9 33.3% Junior 9 33.3% Senior 3 11.1% Race/Ethnicity White 5 18.5% Hispanic or Latino 4 14.8% Asian/Pacific Islander 11 40.7% Black or African-American 6 22.2% International 1 3.7% College Applied health sciences 1 3.7% Architecture, Design & Arts 1 3.7% Liberal arts & Sciences 16 59.3% Business Administration 3 11.1% Engineering 2 7.4% Education 2 7.4% Nursing 2 7.4% 458 College & Research Libraries May 2022 “quiet study space” (59.3%), followed by collaborative study space (25.9%), and “computer areas” (14.8%). None of the students selected “group study room” as a response. Students reported that on average, they stayed in the library between 1.5 hours to 2 hours (25.9%), while some students stayed in the library more than 3 hours (18.5%) a day. The majority of students reported using the library website for less than 30 minutes (73.1%) a day. Students’ Library Visits (In Person and Online) by GPA. Students were grouped by their GPA: Group 1 with less than 3.0, Group 2 between 3.0 and 3.49, and Group 3 with 3.5 or higher. The percentages of students’ in-person library visits and online visits were compared by GPA group. As shown in figure 1, the results indicated that Group 2 (GPA between 3.0 and 3.49) most frequently visited the library in person. However, Group 3 (GPA 3.5 or higher) was the primary reporter of using the online library most frequently. Some users of Group 1 and Group 2 reported “Never” using the online library. Learning Outcomes of Spaces, Website, Instruction. Students were asked to respond with the level of agreement in the learning outcomes questions related to library space, library web- site, and library instruction. As shown in figure 2, the majority of students (92%–100%) rated “somewhat,” “agree,” “strongly agree” in the learning outcomes of using library spaces. Two students responded “disagree” in the “The library is a place to work with my classmates.” Only one student replied “strongly disagree” in the outcome question of “the library space contributed to my ability to learn something new.” All the students responded “agree” and “strongly agree” to their likelihood of using the library space again. With respect to the results of learning outcomes questions related to library website, for most questions the respondents rated “somewhat,” “agree,” or “strongly agree.” One student rated “strongly disagree” in the learning outcomes (knowledge, confidence, and awareness), whereas two students rated “strongly disagree” in the area of application of library website: “I intend to apply library resources to my coursework or research.” FIGURE 1 Students Library Visits (In person vs. Online) by GPA (N = 27) Note: Some columns may not total 100% due to rounding Impact of Undergraduate Students’ Library Use on their Learning beyond GPA 459 FIGURE 2 Learning Outcomes for Library Space (N = 27) Note: Some columns may not total 100% due to rounding FIGURE 3 Learning Outcomes for Library Website (n = 26)* *Note. One participant never used the library website before. Some columns may not total 100% due to rounding 460 College & Research Libraries May 2022 Six out of 27 participants had never attended a library instruction session. As such, a total of 21 participants responded to the learning outcomes for library instruction. The results of learning outcomes questions relevant to library instruction were similar to those of library space and library website where, for most questions, the respondents rated “somewhat,” “agree,” or “strongly agree.” Two students rated “strongly disagree” in the learning outcomes (confidence, awareness, and application), whereas three students rated “strongly disagree” in the area of knowledge: “I learned something new that will help me success in my classes.” FIGURE 4 Learning Outcomes for Library Instruction (n = 21)* *Note. 6 out of 27 participants never attended any library instruction session. Some columns may not total 100% due to rounding FIGURE 5 Students’ Attitudes toward the Library Some columns may not total 100% due to rounding. Impact of Undergraduate Students’ Library Use on their Learning beyond GPA 461 Students’ Attitude toward the Library. Students were asked to rate their level of agreement in their perceptions of how they feel about the library. Overall, most respondents answered “agree” and “strongly agree” in the areas of “I feel comfortable staying in the library” and “I feel welcome,” whereas one respondent rated “agree” in the areas of “I feel I need to make myself fit in the library” and “I feel like an outsider in the library.” Qualitative Findings The participants were further interviewed about what motivated them to study in the library and how they perceived the impact of library use on their learning and academic success. Four major themes emerged from the qualitative analysis of the interviews: 1) students’ motiva- tions for studying in the library; 2) students’ perceptions of library resources use; 3) students’ perceptions of the relationship between library use and their GPA; and 4) students’ percep- tions of their academic success and goals. To support themes derived from the interviews, direct quotes from participants were used. To protect the confidentiality of participants, broad demographic information is provided. Each theme is addressed below. Theme 1: Students’ motivations for studying in the library. This theme refers to par- ticipants expressing their motivations to study in the library rather than other places. Many participants said their primary reason to use the library is to focus and avoid any distraction, get things done, and increase productivity. This pattern emerged regardless of gender and their GPAs. Another primary reason to study in the library was that observing others studying motivated them to study in the library and to not feel lonely. That is, studying in the library al- lowed them to see others’ study behavior in the library, which motivated them to study harder and use the library. One female student who maintained a perfect GPA score commented: “…when I’m like really stressed to study it’s like nice like if I’m at the library and there’s like other people studying, too because then it makes you feel like you’re not alone. Because like you feel like sometimes when you’re studying so much that you’re just so, I don’t know like sometimes I get like sad because this is all I’ve been doing for like I don’t know, like I haven’t done anything like for me recently and I’m just studying, studying. And then you see like other people and it’s like okay I’m not alone, there’s like other people also in the same position.” (Female, Junior, Math, 4.0 GPA) Her motivation to use the library, by observing others studying hard, was to ensure that she would not be studying hard all alone and would remain motivated to continue studying by herself. Another student shared similar reasons for using the library: getting work done and becoming motivated to study in the library by observing others doing the same. “Honestly I feel like when I come to library, I feel like studying and I feel like I can do something. I know it’s kind of a personal reason maybe some people come for fun or some people come for a specific other reason, but whenever I’m doing something in library I feel like I have done something. I feel like I know, in myself I’ve accomplished something but I feel good about myself when I’m studying in the library and I see everyone studying and everyone is focused towards the similar goal, that is to succeed and all, so that’s what I like.” (Male, Sophomore, Engineering, 3.3 GPA) 462 College & Research Libraries May 2022 Theme 2: Students’ perceptions of library resources use. Students’ perceptions of library resources use refers to how and why students used the library resources and how they found out about those resources. While not all participants used the library resources (journals, da- tabases, or books), many participants reported that they primarily relied on the journals or databases to write a paper for their assignments. The use of these resources is often related to the courses they are taking. When participants, primarily in their freshman year, were taking English, history, or anthropology classes, they were required to use databases or journals to write a paper. In contrast, students in the STEM fields such as engineering, computer science, and math did not have a writing assignment and instead were required to take exams or quiz- zes. These students were less likely to report using library resources. For example, a female sophomore student who majored in nursing commented: “I didn’t really use the library website last semester because last semester I was taking Bio, Bio Lab, Anatomy, and Lifespan and Development which was like a nursing or a pre-nursing specific course. So I was taking those classes and those, oh I was taking Psych 242 as well. Those classes I didn’t, it wasn’t really required to write papers or even in Psych there was like a few small writing assignments but nothing that I ever had to use the library resources for.” (Female, Sophomore, Nursing, 3.7 GPA) Only some participants reported using books; some students used books for their assign- ments, whereas the others used books for fun or entertainment. While many of the participants reported that they learned about the library resources primarily through library instruction workshops embedded in English freshman classes, some participants found out through the freshman orientation or a professor. A couple of participants mentioned their siblings, who were enrolled at the same institution, told them how to find library resources. Theme 3: Students’ perceptions of the relationship between library use and their GPA. Almost every participant strongly believed that their use of library space was positively as- sociated with their GPA, independent of how often they would use the library. More spe- cifically, they saw the quiet library space as a positive environment that helped them focus, reduce distractions, and get work done or that motivated them to study harder by seeing others studying hard, which in turn resulted in their higher GPA. For instance, a male junior student explained how his use of the library has an impact on his GPA: “Yes definitely. So like I said it does, in a way the library kind of influences my mindset on the things they need to do. So just helping me stay focused and providing a space where I can be focused and not be distracted definitely helps my grades. Because if I wasn’t focused at all, then I wouldn’t be able to study and then I wouldn’t be able to do all of these things. So the library does help a lot in my academic success.” (Male, Junior, Biology, 3.1 GPA) Other students provided further explanations on the relationships between frequency of library use and their GPA. “Like when I’m, the semesters when my GPA was down, it was when I wasn’t really utilizing the library when I would study at home or like because I’d get distracted or I Impact of Undergraduate Students’ Library Use on their Learning beyond GPA 463 wouldn’t even study at all. So definitely the more time I spend here, the more productive I am and the more I get done, the more I learn, the more I retain through studying so definitely like this is definitely a good place. It’s not distracting at all, at least for me, I learn more, I get more done, and I study more at the library. So the semesters I’ve done well have been, usually been because I’ve been studying at the library.” (Male, Sophomore, Engineering, 2.6 GPA) While most participants primarily focused on the library space use and their GPA, one student commented how the library resources have an impact on her GPA. A female junior student noted: “Yeah, the space definitely because like you study with like your peers or you study with people that are older and you see that how they’re working hard makes you get motivated. The library like website it does help in like research faster instead of going to Google and seeing if like I could use it or if it’s reliable. Because I know most of the article or all of the articles are reliable. And usually with my papers, you need like a work cited, and it’s already there, like work cited, abstract, it’s click and great.” (Female, Junior, Integrated Health Science, 2.5 GPA) Interestingly, a female sophomore student interpreted library use is a mediation of her learning skills and GPA and elaborated: “For me it’s not about my GPA because I feel like my GPA is reflected by my learning skills so I guess in a chain it starts by me going to the library and then the library being a good place to improve my learning skills, and then my learning skills helping me improve my GPA. So I see it like that. I never see it skipping over that learning skills aspect.” (Female, Sophomore, Undeclared, 3.3 GPA) As shown above, most of the participants who expressed the positive impact of their li- brary use on academic success reported a GPA lower than 3.3. This finding suggests that the frequency of library use does not always correlate with the students’ academic grades and, more importantly, students perceive their library use as having a positive impact on their grades. Theme 4: Students’ perceptions of their academic success and goals. Students shared their perceptions of their academic success and goals. While many participants defined academic success as getting a good grade such as a 4.0 GPA, some remarked that, in addition to getting a high GPA, academic success means developing application skills. For example, a male junior student who majored in biology commented: “That means two things for me—a high GPA and it also means that I actually learn the things that I am taught and that I could apply it to my everyday life, or at least I could apply it to the things that I need to do after college. Those are my two things on academic success. I feel like it’s useless to just memorize everything and then forget them after. I like applying what I’ve learned. I know I wouldn’t be able to apply it in every situation, but as long as let’s say my physics class, I see a lot of things here in the real world and 464 College & Research Libraries May 2022 I’m just, now I know kind of the mechanisms behind it and it’s just, it’s interesting to know.” (Male, Junior, Biology, 3.1 GPA) Similarly, others also commented: “learning something new that you can apply to your everyday life,” “understanding what you’re studying and in the future applying it,” “caring about what you’re doing and what you’re learning and actually attempting to learn outside of that class,” and “learning about things that are [exciting] or interesting.” Some participants characterized academic success as the accomplishment of the learning process and obtaining subject knowledge. Examples include “getting work done,” “working hard and getting the results you want,” “feeling confident in the subjects that you are studying and being able to retain it,” being able to discuss it with others,” “try my best and understand the materials.” Only one participant defined academic success as developing interpersonal skills: for example, “being well-rounded and having good soft skills, knowing how to talk to people.” Participants also expressed their academic goals. Many students reported their primary goal is to graduate, while others reported it was going to medical school or getting an oc- cupation. Some participants commented that increasing their GPA was their academic goal. Discussion and Future Directions This paper employed a mixed-methods approach: 1) to measure library use and its impact on stu- dent learning outcomes and GPA using quantitative data and 2) to explore in depth how students’ perceptions of their library use (in person and online) impacts their GPA using qualitative data. A key finding from the quantitative data demonstrated that students with a GPA between 3.0 and 3.49 most frequently visited the library in person. However, those with a GPA 3.5 or higher primarily reported using the online library most frequently. Some users with a GPA lower than 3.0 and those with a GPA between 3.0 and 3.5 reported “Never” using the online library. This finding replicates the previous quantitative study conducted by the authors, in- dicating that students’ online use was positively associated with their GPA, whereas physical library use was negatively associated with their GPA.25 Regarding how the respondents perceived the impact of library space, the library web- site, and instruction on their learning outcomes, the authors discovered that library space has significantly contributed to students’ productivity and confidence (see figure 1). Addition- ally, all the respondents reported that they will return to use the space again. With respect to learning outcomes of library instruction, 90 percent of the respondents reported that library instruction impacted on their confidence, awareness, and application, whereas three students rated “strongly disagree” in the area of knowledge: “I learned something new that [helped] me succeed in my classes.” During the interviews, the respondents provided further explana- tions on their responses. For example, one respondent who selected “strongly agree” further elaborated that “a couple of my assignments like they required like you can’t just look up a Google site…before going into the classes I knew that UIC had access to the databases, but I wasn’t too sure how to access them. So the library instruction course was helpful in show- ing how to access those databases” (Male, Sophomore, Engineering, 2.6 GPA). On the other hand, one student further explained why he selected “strongly disagree,” stating, “Because I already know that they got resources and everything. I might learn something new for my class, like if it’s not assigned, but I don’t think you can learn anything new” (Male, Senior, Marketing, 3.1 GPA). Impact of Undergraduate Students’ Library Use on their Learning beyond GPA 465 Another key finding from this study was that there is a gap between students’ actual library use and recorded GPA (quantitative data) and students’ perceptions of their library use and their GPA (qualitative data). That is, it appeared that their actual frequency of library use was not associated with their reported GPA, whereas most of the students perceived that their use of the library positively influenced their GPA. It is possible that, when examining the impact of library use on students’ GPA, mostly an accumulated GPA is used to measure the correlations between library use and GPA. Given that GPA reflects the accumulation scores of academic years, using an accumulated GPA may not fully uncover how students’ use of the library relates to their GPA per semester. Instead, this study attempted to measure the frequency of library use per semester and their GPA from the beginning of the semester to the current academic year. For example, a male sophomore who had a 2.6 GPA explained that, when he did not use the library, his GPA went down; however, his GPA went up after more frequent use of the library. His 2.6 GPA was an accumulated score since freshman year. However, his comment implies that his GPA was lower when he did not use the library and it is possible that his GPA was higher when he used the library more. While we cannot confirm whether this speculation is accurate, this finding may explain the argument raised by Robertshaw and Asher that previous studies measuring library impact on students’ academic success were not statistically convincing.26 This is also supported by a recent study conducted by Mayer et al. indicating that library services were found to minimally impact final GPAs.27 Nonetheless, comparing students in different GPA ranges is a unique contribution to the current literature. Qualitative findings revealed that students’ perceptions of academic success are not only about getting a higher GPA but also developing applicable skills, accomplishing the learning process, and obtaining subject knowledge. This finding is supported by the previous study indicating that undergraduate students perceived their academic success as applying new knowledge to circumstances in real life and gaining new knowledge.28 This finding has impli- cations for future research. When examining the library’s impact on students’ success, GPA alone is not sufficient. Instead, incorporating outcome variables such as knowledge (obtaining something new) and applications (applying new knowledge into practices) into measuring impact on academic success will provide a more comprehensive picture of how students’ li- brary use has impacted their academic success. Measuring learning outcomes (knowledge and applications) are already included in the Project Outcome Model,29 and this finding provides evidence highlighting the importance of measuring learning outcomes in addition to GPA to have a comprehensive view of library impact and value. Nonetheless, it is still not clear how students perceive their own academic success. Do they consider that they are academically successful? Because an individual’s perception of their own success is subjective and varies by individual. For example, a student who majors in Nursing and earned 3.0 GPA may consider herself or himself successful, while a student who studies English with the same GPA may not perceive himself or herself as successful. A potential research question for the future is this: is library use more highly related to their own perception of being successful? Given that previous studies have predominantly focused on using the GPA as an outcome variable, this study questioned previous findings and attempted to examine students’ academic success with their own perspectives through a mixed-methods approach by using both quantita- tive and qualitative datasets. The findings from the current study will prompt other researchers who are interested in demonstrating the library value from a different angle beyond the GPA. 466 College & Research Libraries May 2022 The authors plan to include the questions “how do you define academic success” and “whether students perceive themselves as successful, and how and why” in future student surveys that target the institution’s entire population. As discussed earlier, examining the correlations between students’ library use and their GPA does not always generate the sig- nificant outcomes academic libraries would like to see. Connecting students’ library use with the academic success that they value and defining their success in terms such as improving grades, completing courses, or obtaining applicable skills will more accurately and directly reflect the library’s value. If academic libraries cannot demonstrate their role in students’ academic success to various stakeholders, budgets for maintaining collections, hiring librar- ians, and providing services could suffer. The current research project is an example of an institution conducting ongoing assessment. This is beneficial for other academic libraries in understanding the potential impact of the academic library on student success and in learning how an academic library can implement ongoing assessment to measure the library’s impact on students’ learning and academic success. This exploratory study with a mixed-methods approach provides evidence of student perceptions not previously captured in the literature. It has potential to shape ongoing and future research in this area. Conclusion Through both quantitative and qualitative data, the current study provided better mechanisms to understand the process of how library has an impact on their learning outcomes as well as GPA. By looking at both sets of data, the authors found that the frequency of library use does not always correlate with the students’ academic grades; more importantly, students perceive their library use as having a positive impact on their grades. It is plausible that GPA is an accumulated outcome based on several academic years, while frequency of library use is measuring activity during a certain time period, resulting in nonexistent relationships between library use and GPA or minimal evidence of impact on grades, such as has been seen in previ- ous studies. Future research is needed to further examine this issue. Additionally, students considered achieving a higher GPA, developing applicable skills, accomplishing the learning process, and obtaining subject knowledge was academic success. These findings provide criti- cal implications for what outcome variables are important when assessing the library values on students’ academic success. While some students voiced that getting a higher GPA was an indicator of academic success, others articulated that applying new knowledge into practice and obtaining knowledge were the characteristics of academic success. To holistically measure the library value on their academic success, future research should consider including these outcome variables in addition to GPA. Acknowledgments The authors thank the University of Illinois Chicago Library Research Fund and their col- league Paula Dempsey for her review and feedback on this article. Impact of Undergraduate Students’ Library Use on their Learning beyond GPA 467 APPENDIX A. Recruitment Survey University of Illinois Chicago (UIC) Library research team invites UIC undergraduate students to join our research study. This study aims to examine students’ primary reasons for using the university library and measure whether students’ library use has an impact on their learning outcomes. To volunteer to participate in this study, please complete the screening survey below. Your responses to the survey questions below will be kept confidential and will be used to select participants for this study. Your response to this survey will be retained until subject recruitment has been completed. Once recruitment is complete, your responses to this survey will be destroyed. If you are selected to participate, a member of the research team will contact you for an interview. You will receive $25 for your participation in the interview. 1. During the last semester, did you do any of the following at the UIC Library? Select all that apply. □ Visited the UIC Library in person □ Used the online UIC Library □ None of above—Skip to end of survey: Thank you for your interest in our research study. 2. Are you a UIC undergraduate student? □ Yes □ No—Skip to end of survey: Thank you for your interest in our research study. 3. How often did you visit the UIC Library in person last semester? □ Never □ Once a month □ Once a week □ Multiple times in a week □ Daily 4. How often did you use the online library last semester? □ Never □ Once a month □ Once a week □ Multiple times in a week □ Daily 5. What was your GPA last semester? □ Less than 3.0 □ 3.0–3.5 □ Greater than 3.5 Please provide your name and contact information. If you are selected to participate, you will receive $25 for your time at the end of the interview. Name: UIC email: Phone number: 468 College & Research Libraries May 2022 APPENDIX B. Pre-Interview Survey Questions (In-person visit) I usually come to the library for___________________________ Please select only THREE primary reasons to use the library. □ studying alone □ studying with friends □ working on assignments □ using print library material to work on an assignment □ using online library material to work on an assignment □ working on group projects with my classmates □ checking out books/returning books □ looking for materials □ attending workshops/sessions hosted by the UIC Library □ using computers or printers for assignments or preparing for exams □ using computers or printers for non–course-related purposes □ asking reference questions to librarians □ attending UIC Library instruction sessions taught by a librarian (such as how to find resources) □ spending time with my friends □ spending time between classes □ stopping by the cafe □ using restrooms/drinking water □ passing through to go to other buildings □ others________________ (Online use) I use the library for__________________________________. (Please select one) □ accessing library e-books, journals etc. to help me complete assignments □ accessing library e-books, journals, etc. to help me study □ accessing library e-books, journals, etc. to help expand my knowledge outside of course assignments □ requesting library materials (books, journals) from other libraries □ using “chat with a librarian” □ other ________________ (In-person visit) When you come to the library, which space do you use the most? (Please select one) □ First floor (circle reading room) □ First floor (IDEA commons) □ Second floor □ Third floor □ Fourth floor (In-person visit) When you come to the library, which space do you use the most? (Please select one) □ Quiet study space □ Collaborative study space Impact of Undergraduate Students’ Library Use on their Learning beyond GPA 469 □ Group study room □ Computer area (In-person visit) On average, how long do you usually stay in the library? (Please select one) □ less than 30 minutes □ 30 minutes–less than 1 hour □ 1 hour–less than 1.5 hours □ 1.5–less than 2 hours □ 2 hours–less than 2.5 hours □ 2.5 hours–less than 3 hours □ more than 3 hours (Online use) On average, how long do you usually spend time on the library website? □ ________minutes Learning Outcomes Do you agree or disagree with the following statement? 1. Strongly disagree, 2. Disagree, 3. Somewhat, 4. Agree, 5. Strongly agree (In-person visit: Library space) □ The library space contributed to my ability to learn something new □ Using this space makes me feel more confident about my ability to achieve my goals □ I am likely to use this space again in the future □ After using this space, I am more aware of the library resources and services avail- able to me □ The library is a place to work with my classmates □ The library environment helps me stay focused (In-person: Library instruction) After attending the library instruction and/or asking reference questions to library, □ I learned something new that will help me succeed in my classes □ I feel more confident about completing my assignments □ I intend to apply what I just learned □ I am more aware of the library’s resources and services (In-person visit: Students’ attitude toward the library) □ I feel welcome □ I feel comfortable staying in the library □ I feel I need to make myself fit in the library □ I feel like an outsider in the library (In-person visit: Competency) □ I get what I need at the library □ I easily get lost in the library (Online use: Library website) □ I learned something new that will help me succeed in my classes □ I feel more confident about completing my assignments □ I intend to apply library resources to my coursework or research □ I am more aware of the library’s resources and services 470 College & Research Libraries May 2022 What is your age? □ 18–25 years old □ 26–35 years old □ 36–45 years old □ 46–55 years old □ 56 years or older What is your gender? □ Female □ Male □ Other (specify) What college are you in? □ Engineering □ Liberal Arts & Sciences □ College of Medicine □ Pharmacy □ Education □ Nursing □ Applied Health Sciences □ Business Administration □ Dentistry □ Social Work □ Architecture, Design & Arts □ Urban Planning & Public Affairs □ Graduate College □ School of Public Health Please specify your ethnicity □ White □ Black or African-American □ Asian/Pacific Islander □ Hispanic or Latino □ International □ Other Do you have a computer at home? □ Yes □ No Do you have internet access at home? □ Yes □ No Do you live on the campus? □ Yes □ No Impact of Undergraduate Students’ Library Use on their Learning beyond GPA 471 How far do you live from [University Name] campus? □ Less than 1 mile □ 1–5 miles □ 6–10 miles □ 11–15 miles □ 16–20 miles □ 21 miles or further Do you have your own study space at your house? □ Yes □ No Are you currently working in the UIC library? □ Yes □ No What is your GPA? [ ] 472 College & Research Libraries May 2022 APPENDIX C. Interview Protocol Thank you for agreeing to participate in the interview. I am going to ask questions about your library experiences. 1. You completed the survey and indicated that you use the library for___________. • Can you tell me more about why you visit or use the library? • Do you have any other reasons you visit or use the library? 2. What motivates you to attend library instruction sessions? • What did you like most about the library instruction? • What content of the library instruction helped you increase grades or enhance your research project? 3. Tell me your experience with library staff. • How is your overall experience with library staff? 4. Tell me how you use the library space. • Why do you use the library space? • What types of work do you do in the library? (for instance: assignment? group project? research project? preparing for exams or others?) • How is your overall experience using the library spaces? 5. Tell me what you think of the library environment. 6. Tell me what you think of the library resources. • What types of library materials do you usually rely on? 7. What does the library mean to you? • What did you like most about the library? 8. Do you have any challenges when using the library? If yes, can you tell me more about it? 9. Tell me about your experiences with the library and how that impacted your grades, coursework or research. • What could the library do to help you succeed in your classes? 10. We talked about several topics today. Are there any comments? Notes 1. Jung Mi Scoulas and Sandra L. De Groote, “The Library’s Impact on University Students’ Academic Suc- cess and Learning,” Evidence Based Library and Information Practice 14, no. 3 (2019): 2–27, https://doi.org/10.18438/ eblip29547. 2. Karen Brown and Kara J. Malenfant, “Academic Library Impact on Student Learning and Success: Find- ings from Assessment in Action Team Projects,” in Shaping the Campus Conversation on Student Learning and Experience: Activating the Results of Assessment in Action, eds. Karen E. Brown et al. (Chicago, IL: Association of College & Research Libraries, 2018), 105–32; Lisa Massengale, Pattie Piotrowski, and Devin Savage, “Identifying and Articulating Library Connections to Student Success,” College & Research Libraries 77, no. 2 (2016): 227–35, https://doi.org/10.5860/crl.77.2.227 3. Scoulas and De Groote, “The Library’s Impact on University Students’ Academic Success and Learning.” 4. Deeann Allison, “Measuring the Academic Impact of Libraries,” portal: Libraries and the Academy 15, no. 1 (2015): 29–40, https://doi.org/10.1353/pla.2015.0001; Tiffany LeMaistre, Qingmin Shi, and Sandip Thanki, “Connecting Library Use to Student Success,” portal: Libraries and the Academy 18, no.1 (2018): 117–40, https://doi. org/10.1353/pla.2018.0006; Krista M. Soria, Jan Fransen, and Shane Nackerud, “Beyond Books: The Extended Academic Benefits of Library Use for First-Year College Students,” College & Research Libraries 78, no. 1 (2017): 8–22, https://doi.org/10.5860/crl.v78i1.16564; Francesca Marineo and Qingmin Shi, “Supporting Student Success https://doi.org/10.18438/eblip29547 https://doi.org/10.18438/eblip29547 https://doi.org/10.5860/crl.77.2.227 https://doi.org/10.1353/pla.2015.0001 https://doi.org/10.1353/pla.2018.0006 https://doi.org/10.1353/pla.2018.0006 https://doi.org/10.5860/crl.v78i1.16564 Impact of Undergraduate Students’ Library Use on their Learning beyond GPA 473 in the First-Year Experience: Library Instruction in the Learning Management System,” Journal of Library & Information Services in Distance Learning 13, no. 1/2 (2019): 40–55, https://doi.org/10.1080/1533290x.2018.1499235. 5. Association of College & Research Libraries (ACRL), “Academic Library Impact: Improving Practice and Essential Areas to Research,” prepared by Lynn Silipigni Connaway, William Harvey, Vanessa Kitzie, and Stephanie Mikitish of OCLC Research (Chicago, IL: ACRL, 2017), www.ala.org/acrl/sites/ala.org.acrl/files/content/ publications/whitepapers/academiclib.pdf. 6. Ula Gaha, Suzanne Hinnefeld, and Catherine Pellegrino, “The Academic Library’s Contribution to Student Success: Library Instruction and GPA,” College & Research Libraries 79, no. 6 (2018): 737–46, https://doi.org/10.5860/ crl.79.6.737. 7. LeMaistre, Shi, and Thanki, “Connecting Library Use to Student Success.” 8. Marineo and Shi, “Supporting Student Success in the First-Year Experience.” 9. Travis T. York, Charles Gibson, and Susan Rankin, “Defining and Measuring Academic Success,” Practical Assessment, Research & Evaluation 20, no. 5 (Mar. 2015): 1–3. 10. M. Brooke Robertshaw and Andrew Asher, “Unethical Numbers? A Meta-Analysis of Library Learning Analytics Studies,” Library Trends 68, no. 1 (2019): 76–101, https://doi.org/10.1353/lib.2019.0031 11. “ACRL Benchmark,” https://www.ala.org/acrl/proftools/benchmark/. 12. American Library Association (ALA), “Project Outcome for Academic Libraries: A Year in the Field,” American Library Association (June 3, 2020), www.ala.org/news/member-news/2020/06/project-outcome-aca- demic-libraries-year-field. 13. Sara Goek, “Project Outcome for Academic Libraries” (Chicago, IL: American Library Association, June 2020), https://acrl.projectoutcome.org. 14. Jennifer Mayer et al., “Undergraduate Student Success and Library Use: A Multimethod Approach,” Col- lege & Research Libraries 81, no. 3 (2020): 378–98, https://doi.org/10.5860/crl.81.3.378. 15. Mayer et al., “Undergraduate Student Success and Library Use.” 16. John W. Creswell and Cheryl N. Poth, Qualitative Inquiry & Research Design: Choosing Among Five Approaches, 4th ed. (Thousand Oaks, CA: Sage, 2018). 17. Beyza Aksu Dunya and Sandra L. De Groote, “Revision of an Academic Library User Experience Survey,” Performance Measurement and Metrics 20, no. 1 (2019): 48–59, https://doi.org/10.1108/PMM-10-2018-0026. 18. ALA, “Project Outcome for Academic Libraries.” 19. ALA, “Project Outcome for Academic Libraries.” 20. Paula J. Lavrakas, Encyclopedia of Survey Research Methods (Thousand Oaks, CA: Sage, 2008), https://doi. org/10.4135/9781412963947. 21. Cathy Urquhart, Grounded Theory for Qualitative Research: A Practical Guide (Thousand Oaks, CA: Sage, 2013). 22. Lisa M. Given, 100 Questions (and Answers) about Qualitative Research (Thousand Oaks, CA: Sage, 2016). 23. Hsiu-Fang Hsieh and Sarah E. Shannon, “Three Approaches to Qualitative Content Analysis,” Qualitative Health Research 15, no. 9 (2016): 1277–88. 24. Hsieh and Shannon, “Three Approaches to Qualitative Content Analysis.” 25. Scoulas and De Groote, “The Library’s Impact on University Students’ Academic Success and Learning.” 26. Robertshaw and Asher, “Unethical Numbers?” 27. Mayer et al., “Undergraduate Student Success and Library Use.” 28. Mayer et al., “Undergraduate Student Success and Library Use.” 29. ALA, “Project Outcome for Academic Libraries.” https://doi.org/10.1080/1533290x.2018.1499235 http://www.ala.org/acrl/sites/ala.org.acrl/files/content/publications/whitepapers/academiclib.pdf http://www.ala.org/acrl/sites/ala.org.acrl/files/content/publications/whitepapers/academiclib.pdf https://doi.org/10.5860/crl.79.6.737 https://doi.org/10.5860/crl.79.6.737 https://doi.org/10.1353/lib.2019.0031 https://www.acrlmetrics.com/ http://www.ala.org/news/member-news/2020/06/project-outcome-academic-libraries-year-field http://www.ala.org/news/member-news/2020/06/project-outcome-academic-libraries-year-field https://acrl.projectoutcome.org https://doi.org/10.5860/crl.81.3.378 https://doi.org/10.1108/PMM-10-2018-0026 https://doi.org/10.4135/9781412963947 https://doi.org/10.4135/9781412963947