382 Search Strategy Development in a Flipped Library Classroom: A Student-Focused Assessment Michael C. Goates, Gregory M. Nelson, and Megan Frost* Librarians at Brigham Young University compared search statement de- velopment between traditional lecture and flipped instruction sessions. Students in lecture sessions scored significantly higher on developing search statements than those in flipped sessions. However, student evaluations show a strong preference for pedagogies that incorporate elements from both lecture and flipped methodologies. Reasons for lower flipped-session scores may include a lack of student accountability, strong preference for a live demonstration, and disconnections between online tutorial content and in-class collaborative activities. Librarians using a flipped classroom should consider ways to help students make meaningful connections between online tutorials and in-class activities. Many institutions of higher education are using a flipped classroom teaching model to provide enhanced learning opportunities for students.1 This model moves traditional lecture material to be completed as homework (for instance, reading assignments and video-recorded lectures), reserving class time for hands-on application and collabora- tive activities.2 Multiple studies in various academic disciplines have compared student learning outcomes between the flipped classroom and traditional lecture methodologies in credit-bearing courses. Results from these studies have generally shown improved learning outcomes for students in a flipped classroom environment. The flipped classroom model can also be used in noncredit-bearing single-session workshops, such as library instruction sessions. Implementation of this model for li- brary instruction necessitates the creation of instructional materials that students can complete prior to the library session, such as online library tutorials or video record- ings. Many librarians have compared the effectiveness of online tutorials to face-to-face instruction. While results vary, most studies have found that online tutorials are equally as effective as face-to-face instruction for student comprehension of library instruction concepts.3 Consequently, many academic librarians have incorporated online tutorials * Michael C. Goates is a Life Sciences Librarian in the Harold B. Lee Library at Brigham Young University; e-mail: michael_goates@byu.edu. Gregory M. Nelson is a Chemical and Life Sciences Librarian in the Harold B. Lee Library at Brigham Young University; e-mail: greg_nelson@byu.edu. Megan Frost is a Physiological Sciences Librarian in the Harold B. Lee Library at Brigham Young University; e-mail: megan@byu.edu. ©2017 Michael C. Goates, Gregory M. Nelson, and Megan Frost, Attribution-NonCommercial (http:// creativecommons.org/licenses/by-nc/4.0/) CC BY-NC. doi:10.5860/crl.78.3.382 mailto:michael_goates@byu.edu mailto:greg_nelson@byu.edu mailto:megan@byu.edu http://10.5860/crl Search Strategy Development in a Flipped Library Classroom 383 into their flipped library instruction sessions.4 Several authors have discussed their experiences using this model and offer ideas for implementation.5 These reports are primarily anecdotal and lack concrete assessment of student performance. Because library instruction often lasts for only a single session, a flipped library classroom may have unique implementation and assessment challenges when compared to a credit-bearing semester-long course. Review of Selected Literature Many researchers using quasi-experimental study designs have observed improved student learning outcomes in flipped undergraduate courses. For example, Christopher Mortensen and Angie Nicholson implemented a flipped classroom in an introductory animal science course by recording live lectures and making these available to students online.6 During class time, students participated in a variety of learning activities to reinforce content delivered in the recorded lectures. When compared to traditional lecture sections from prior semesters, students in the flipped classroom scored signifi- cantly higher on general course and critical thinking exams. In a similar study, Kathy Missildine and colleagues found that undergraduate nursing students in a flipped classroom had significantly higher exam scores than students in a traditional lecture.7 However, student satisfaction in these sections was significantly lower than in lecture sections. This underscores the importance of considering both student performance and satisfaction, as these facets can be independent of each other. Other studies have used a modified flipped design to analyze student learning outcomes. For example, Cheryl Talley and Stephen Scherer implemented a partially flipped instructional design for a physiological psychology course in an effort to in- crease student engagement in course material and to apply higher level thinking skills.8 In this study, researchers used a hybrid model to deliver course material (25% flipped, 75% lecture). When compared to previous semesters using a 100 percent traditional lecture approach, students in the hybrid classroom had significantly higher final grades. Though lacking a true control group, the results from this study indicate that even flipping a smaller amount of course material can result in positive student learning outcomes. Similarly, Jack Eichler and Junelyn Peeples found that incorporating a blend of flipped and lecture sessions (four flipped, thirteen lecture) in a single section of a general chemistry course resulted in significantly higher final grades when compared to a lecture-only section.9 In this study, student satisfaction with the instruction meth- odology and learning experience was nearly indistinguishable between the partially flipped and lecture-only sections. It is important to note that, while final grades were higher in the partially flipped section of this study, the final exam scores were not statistically different between the two treatment groups. The higher final grades were primarily attributed to improved clicker assignment scores in the flipped section. It is possible that flipped instruction has greater impact on short-term rather than long- term learning gains, though more research is needed to explain these relationships. Despite the overall positive findings from these studies, there still are uncertain- ties with the flipped classroom. In a scoping review of flipped classrooms in higher education, Jacqueline O’Flaherty and Craig Phillips explained that, while most studies reported improved student learning outcomes and satisfaction in a flipped setting, very few of these studies used robust empirical designs to evaluate higher-order thinking cognitive skills.10 Additionally, this review highlighted the absence of empirical data to determine if long-term improved educational outcomes persist beyond a single course. Similarly, Jamie Jensen and colleagues reported that evaluations of flipped classrooms have typically consisted of either case studies providing anecdotal information on student learning outcomes or comparison studies using quasi-experimental designs 384 College & Research Libraries March 2017 that fail to control for all confounding variables.11 Consequently, they developed a study comparing a flipped model with a nonflipped model in an undergraduate general biology course that attempted to control for all variables except the role of the instructor. The instructor’s role changed from facilitating initial content attainment in the nonflipped model to assisting in concept application in the flipped model. Results from their study found that student attitudes and gains in scientific reasoning ability were equivalent in both flipped and nonflipped methodologies. An active learning environment appeared to be the greatest predictor of student learning outcomes and attitudes, regardless of class structure (flipped vs. nonflipped). Other studies have also found that a flipped methodology has little impact on student learning and preferences. In a study using an undergraduate course in civil engineering, Hotle and Garrow attempted to control for confounding variables by comparing a flipped and traditional lecture course taught during the same semester by the same instructor.12 All other variables in this study were identical, though the traditional section did not incorporate any form of active learning. Student quiz per- formance did not differ significantly between treatments. Likewise, students reported comparable positive experiences with both the traditional and flipped methodology. Similarly, Randall Davies and colleagues found that a flipped learning environment was equally as effective as a traditional lecture model for a five-week introductory- level undergraduate information systems spreadsheet course.13 Student test scores and course evaluations did not differ significantly between teaching methodologies. Results from these studies indicate that other factors may have a greater impact on student performance and attitudes than the teaching methodology. A few empirical studies have attempted to assess the flipped classroom model for library instruction. Karen Anderson and Frances May incorporated a hybrid instruc- tion model using an online tutorial in conjunction with an in-class library instruction session.14 The hybrid model required students to complete an online tutorial prior to attending an in-class library session. They found that students performed equally well at constructing keyword and Boolean searches in face-to-face, online, and hybrid teaching models. However, this study did not test a truly flipped classroom because the in-class session of the hybrid model was identical to the instruction in the face-to- face model and lacked student collaboration and active learning. In a similar study, Andrea Brooks found that students in flipped and traditional lecture sessions performed equally well at identifying search terms, developing research questions, and imple- menting search strategies.15 Limitations from this study included a small sample size and virtually identical in-class learning activities between instruction methodologies. Although the librarian spent less time explaining concepts in the flipped classroom, there is no indication that students were able to use this extra time for more engaged learning activities. Eduardo Rivera found that students in a flipped seven-week library workshop had higher post-test scores than those in a traditional workshop.16 However, the widespread applicability of these findings are uncertain, as these workshops were completed over different semesters and student scores were not compared using tests of statistical significance. Additionally, a seven-week workshop would likely produce different results than a single library instruction session. The results from the studies described above imply that the flipped classroom may be a viable pedagogy for library instruction. However, it is difficult to generalize these results to other library instruction settings. More empirical research is needed to analyze a truly flipped model with engaged learning activities for a one-shot library instruc- tion session. A flipped library instruction session that allows students to participate in collaborative, hands-on application exercises is expected to enhance student learning over traditional lecture and online-only instruction methods. The aim of this study is Search Strategy Development in a Flipped Library Classroom 385 to compare student performance in constructing effective search strategies between a traditional library instruction session and a flipped classroom session that uses an online tutorial. Hypotheses Students in flipped sessions who completed the online tutorial were expected to out- perform students in lecture sessions because they participated in face-to-face collab- orative activities that reinforced the material learned in the online tutorial. Students in flipped sessions who did not complete the tutorial were anticipated to perform worse than those who completed the tutorial as well as students in lecture sessions primarily because they did not receive instruction on conducting searches as either part of the tutorial or a traditional lecture. Timing of the tutorial viewing was expected to impact student performance on developing effective search strategies. Students who viewed the tutorial further in advance of the in-class session were expected to perform worse than those who completed it closer to the time of the in-class session. Students were expected to express a strong preference for the interactivity of a flipped session when compared to the lecture session. Methodology All study methods involving human subjects were approved by the Brigham Young University Institutional Review Board (protocol X14496). Study Participants Participants were undergraduate students, primarily in the life sciences, who were also enrolled in an advanced writing course at Brigham Young University (BYU) dur- ing the winter semester of 2015. The advanced writing course is a general education requirement offered through the BYU English Department. As part of the advanced writing course, students are required to register for a fifty-minute subject-specific li- brary instruction session. Students who registered for a life sciences library instruction session were invited to participate in this study. During the first five minutes of each session, students were informed of the purpose of this research study and were given the option to decline participation. In total, 122 students consented to participate in this study in eight unique sessions. Study Design Each of the eight life sciences library sessions was randomly assigned to one of two treatments: lecture or flipped. The same life sciences librarian taught all eight sessions. Lecture Treatment. The lecture treatment followed a traditional information literacy instruction model. Lecture topics included source evaluation, the peer-review process, keyword searching, truncation, and Boolean operators. This section of the lecture took approximately ten minutes. Following this lecture, the librarian gave a live search dem- onstration of the Web of Science database to find research articles. This demonstration took approximately twenty minutes. Students then had fifteen minutes to complete the session assignment and evaluation. Flipped Treatment. Students in the flipped treatment were e-mailed a link to the online Life Sciences Tutorial (net.lib.byu.edu/tutorial/lifescience) two weeks prior to the scheduled instruction session. Students were instructed to complete the tutorial before attending the library session. Follow-up e-mails reminding students to complete the tutorial were sent one week as well as one day before each scheduled instruction session. The tutorial covered the same topics addressed during the lecture (source evaluation, the peer-review process, keyword searching, truncation, Boolean opera- http://net.lib.byu.edu/tutorial/lifescience 386 College & Research Libraries March 2017 tors, and a Web of Science search demonstration) and required approximately fifteen to twenty minutes to complete. During the in-class session, the librarian omitted the lecture period and began by responding to student questions about the online tuto- rial. During this discussion, the librarian reminded students about keyword search- ing, truncation, and Boolean operators, as explained in the tutorial. This discussion usually lasted about five minutes. Students then had approximately twenty minutes to work with a partner to find sources on their own topics to be used in an assigned research paper for their advanced writing course. Students were instructed to explain their topics to their partners and, as a team, develop an effective search strategy to find relevant articles on these topics. During this exercise, the librarian and teaching assistant briefly visited with each partnership to answer any questions and provide further direction. This hands-on search activity was followed by a five-minute group discussion exploring student successes and struggles with the previous search activity. Students then had fifteen minutes to complete the session assignment and evaluation. The Assignment. Each participant was asked to develop a search strategy on a topic assigned by the life sciences librarian. All participants in both the lecture and flipped treatments were assigned the same topic (“the impact of soft drink consumption on childhood obesity”) and asked to complete the same worksheet (see appendix A). This assignment required students to identify the main concepts of the assigned topic, list relevant search terms and potential synonyms, and develop an effective search strat- egy in the Web of Science database using Boolean operators and database limiters. The worksheet also contained a question asking students if they completed the online tutorial. For those who completed the tutorial, a follow-up question asked students to indicate when they watched the tutorial. Students in both treatments had fifteen minutes to complete this assignment, at which time the worksheets were collected. The completed student assignments were randomly sorted and then graded by two life science librarians (who had not participated in the instruction phase of the study) using a predetermined rubric (see appendix B). This rubric measured the complexity and completeness of each search strategy and the appropriate use of Boolean operators and database limiters (such as relevant search terms, synonyms, truncation, date range, and quotations). Only the final search statement portion of this assignment (ques- tions 4a, 4b, and 4c) was graded, for a total of thirteen points. Prior to grading actual student assignments, the grading librarians were trained to the rubric using practice worksheets to ensure consistency in grading. Both librarians graded each student assignment. When the librarians’ scores for a worksheet were within two points, the average between the two scores was assigned. When the scores differed more than two points, the instruction librarian for this study acted as mediator to determine which of the two scores was more accurate. Overall, the grading librarians had high inter-rater reliability (intraclass correlation coefficient = 0.984). Session Evaluation. At the end of the library session, participants were asked to complete a short evaluation of the session (see appendix C). Students were asked to identify their gender and year in school, allowing the study researchers to determine if performance was impacted by these factors in either of the study treatments. Students were given the opportunity to answer two open-ended questions about what they found to be the most useful about this session and how it could be improved. Analysis Students were categorized into three distinct groups: 1. Students in the lecture treatment (lecture) 2. Students in the flipped treatment who watched the online tutorial (flipped/yes) 3. Students in the flipped treatment who did not watch the online tutorial (flipped/no) Search Strategy Development in a Flipped Library Classroom 387 The flipped/yes group was further categorized by when students reported viewing the tutorial: 1. Day of session 2. One to three days prior to session 3. Four to seven days prior to session 4. More than seven days prior to session Student worksheet grades were analyzed to see if scores differed significantly be- tween groups and if the timing of tutorial viewing in the flipped/yes group had any impact on student worksheet performance. A one-way ANOVA with a Tukey-Kramer post-hoc test was used to compare scores between groups. The Tukey-Kramer post- hoc test is commonly used with ANOVA when comparing three or more groups and identifies significant differences among the means of each group. An independent- samples t-test was used to compare male and female search statement scores in each of the three groups. All statistics were calculated using SPSS Statistics 22 software (IBM Corp., Armonk, New York, USA, 1989–2013). Student comments were coded and tabulated to identify recurring themes. Each comment was analyzed and categorized by theme. These comments were then grouped, tallied, and compared between the two teaching treatments. Results Search Statements Students in the lecture group had higher search statement scores than either the flipped/ yes or flipped/no group. The flipped/no group had lower search statement scores than the other two groups (see table 1). The mean search statement scores differed signifi- cantly among each of the three groups at the 0.01 level (see table 2). Fifty-three of the fifty-four students in the flipped/yes group reported when they viewed the online tutorial. The vast majority of these students watched the tutorial three days or less before the scheduled library instruction session (see table 3). While the mean scores for students who watched the tutorial more than three days before the library session were lower than those who watched the tutorial closer to the scheduled session, the differences between these scores were not significant (all P-values > 0.329). TABLE 1 Mean Search Statement Score by Instruction Group Group Number of Participants Mean Score (out of 13) Standard Deviation Lecture 57 8.78 2.37 Flipped/Yes 54 7.11 2.41 Flipped/No 11 4.7 1.26 TABLE 2 Tukey-Kramer Post-Hoc Comparisons for the Mean Search Statement Score between Instruction Groups Group (I) Group (II) Mean Difference (I–II) Standard Error Lecture Flipped/Yes 1.67422* 0.43990 Lecture Flipped/No 4.07616** 0.76287 Flipped/Yes Flipped/No 2.40194* 0.76629 *P < 0.01; ** P < 0.001 388 College & Research Libraries March 2017 There were no significant differences in search statement scores by gender (all P- values > 0.181) or by year in school (all P-values > 0.564) for any of the instruction groups. Student Evaluations Students provided more positive responses than negative responses for both the lecture and flipped instruction models (lecture: positive 76, negative 44; flipped: posi- tive 98, negative 59). The top two positive responses were the same for both teaching methodologies (learning search strategies and library databases/resources). Figure 1 reports the frequency of all student response categories for both the lecture and flipped instruction models. Discussion The results from this study both validated and refuted various hypotheses concerning student performance with library instruction methodologies. Not surprisingly, students in the flipped treatment who did not complete the online tutorial performed worse than any of the other groups in this study. While they engaged in the same learning activities as others in the flipped treatment, participants in the flipped/no group re- ceived the least amount of instruction on developing effective search strategies. One of the more interesting findings from this study was that students in the lecture group performed significantly better on developing search statements than students in the flipped/yes group. Several factors may be influencing student performance and are worthy of discussion. One possible explanation for the lecture performance advantage is the immediacy of the lecture instruction in relation to the assigned task. Students in the lecture group were given a live demonstration immediately before being asked to perform a database search. For students in the flipped/yes group, more time had transpired between tutorial viewing and performing the database search. This differ- ence in performance may be a function of time. If students in the lecture group were asked to perform a database search twenty-four hours after the instruction sessions, it is possible that these scores would be closer to what was observed in the flipped model. Another factor that may be impacting student performance in this study is the casual completion of the online tutorial for students in the flipped group. Even though the vast majority of students in the flipped classroom (fifty-four of sixty-five participants) viewed the online tutorial prior to attending the in-person session, there is no way of assessing the quality of this viewing experience. If students were multitasking during tutorial viewing (for instance, using social media, reading e-mails, eating, and so on), they easily could have missed key instruction elements. Likewise, students in this study may have lacked sufficient motivation to focus on the material presented in the tutorial. Incorporating a quiz or other assignment as part of the online tutorial may help students to give greater attention to this material. A related consideration is the TABLE 3 Mean Search Statement Score by Timing of Tutorial Viewing for Students in the Flipped/Yes Group Timing of Tutorial Viewing Number of Participants Mean Score (Out of 13) Standard Deviation Day of Session 22 7.16 2.12 1 to 3 Days Prior to Session 24 7.53 2.53 4 to 7 Days Prior to Session 2 5.75 3.89 More than 7 Days Prior to Session 5 5.5 2.72 Search Strategy Development in a Flipped Library Classroom 389 FIGURE 1 Results from Student Session Evaluations Categorized by Theme 390 College & Research Libraries March 2017 human component to instruction. While online tutorials are in many ways convenient for both instructors and students, these tools still cannot entirely replace the influence of face-to-face human interaction. Students may be more compelled to listen closely to a live instructor. Student preferences for library instruction models were more complicated than ini- tially hypothesized. Many comments were similar between the two instruction models, but some differences emerged between the two treatments. For example, many of the negative comments from the lecture treatment were identified as positive aspects of the flipped treatment. This relationship also held true for the negative flipped responses and the positive lecture responses. For example, the most commonly received negative flipped comment related to wanting more search strategy demonstration, while the third most common positive lecture comment related to the modeling of the search process. Similarly, a common negative lecture comment addressed the need for more time for practice, while the third most common positive flipped comment related to student practice and hands-on learning. Several recurring themes appeared important for student learning, regardless of the teaching methodology. First, students valued learning new research strategies and discovering new library resources. Students also appreciated having a live dem- onstration of the search process. Students had a strong desire to have interaction with the library instructor and to receive individualized one-on-one help. Having time to practice what was learned through hands-on application activities also ranked high on student responses. Additionally, students preferred researching their own topics as opposed to searching on an assigned topic. When developing a flipped library instruction session, the instructor should carefully consider the specific goals and desired outcomes. Thorough evaluation of these out- comes will help determine the appropriate instruction method. In our study, we looked specifically at student performance on developing search strategies. It is possible that other information literacy topics would be more amenable in a flipped environment. Library instruction format should also be an important consideration. For example, a flipped classroom model may work better over multiple instruction sessions when compared to a traditional one-shot session because students and instructors are better able to gauge expectations. One important takeaway we found is that an instruction session does not need to be entirely flipped to be effective. Instructors can choose to modify the amount of content that is presented in a flipped methodology. Based on student feedback, some portions of an instruction session may best be taught through live demonstration or lecture while other parts are more effective through collaborative learning activities and hands-on ap- plication. By incorporating multiple teaching methodologies, instructors are able to reach more students with various learning styles. For the material presented in this study, a modification that may prove more effective would require the students to view the online tutorial in advance, followed by an instructor-led demonstration in class. This would reinforce the material while still allowing students to use the majority of the in-class time to work collaboratively and receive personalized assistance from the library instructor. Another consideration with the flipped classroom methodology is the importance of connecting the out-of-class material with the in-class activity. Initially, the flipped treatment in this study did not include the brief five-minute recap of the material covered in the tutorial. After one session, it became apparent that students were not applying the concepts from the tutorial in the database searches as anticipated, even though the librarians felt that the tutorial was rather straightforward. Consequently, the five-minute recap was added for all remaining flipped sessions, and the results from the first session were excluded from this study. This highlights the importance Search Strategy Development in a Flipped Library Classroom 391 of integrating out-of-class learning with in-class activities. Instructors should carefully consider ways to connect both sides of a flipped classroom for their students. Simply asking students to complete a tutorial before coming to class might not provide students the motivation to come prepared for the in-class activity. Ideally, students should be informed how the out-of-class activity will help them in class. For example, the out- of-class activity could get them started on an assignment to be completed in class. Limitations As with any research, this present study has some limitations that should be consid- ered. First, the library instruction used was narrowly focused on developing search strategies. As mentioned earlier, other information literacy topics may provide differ- ent outcomes in a flipped pedagogy. Extrapolating the results from this study to other unrelated instruction settings may not provide accurate representation. Second, the vast majority of students in this study were juniors and seniors (freshmen = 2, sophomores = 7, juniors = 52, and seniors = 60). Even though we did not observe significant differences in search statement scores based on a student’s year in school, our sample clearly had few first- or second-year students. Most of the students in our study have already had some exposure to searching databases and would likely respond differ- ently from a more representative group of first- and second-year students. Third, participants in the study had no real incentive to perform well. Student per- formance on the search statements was not connected to their course grade or standing, likely removing meaningful motivation for many participants. Finally, the search assignment used to evaluate participant performance was some- what artificial. For ease in grading, all participants were given the same search topic. However, search performance would likely improve if students were allowed to select their own topics. Conclusion The results from this study highlight the complexities of implementing a flipped classroom teaching model for a one-shot library instruction session. These complexi- ties include (but are not limited to) maintaining student accountability in completing out-of-class activities and connecting pre-class learning assignments with in-class collaborative activities. These are not insurmountable obstacles to a successful flipped library instruction program, but instructors should give careful thought to mitigating these potential pitfalls. Student preferences are also an important consideration for any library instruction program. A diversified instruction approach, with elements of flipped and traditional teaching methods, appears to address a broader range of learning styles. Future research should focus on finding ways to improve the flipped classroom model for a single instruction session, particularly through increasing stu- dent engagement with out-of-class assignments and coordinating these assignments with in-class learning activities. Additionally, quantitative and qualitative research on other topics beyond search statement development will give a greater understanding of the utility of the flipped instruction model for library instruction. Acknowledgements We would like to thank Holt Zaugg for his expertise and guidance in helping us to develop and administer this study protocol. Also, thanks to Brian Roberts for his as- sistance on some of the technical aspects of this research. We are especially grateful to Leanna Balci and Rita Christensen for their superb work on developing the Life Sciences Tutorial used in this study. Likewise, we are indebted to Dusty Lybbert and Jeremy Nixon for the countless hours they spent creating and editing this tutorial. 392 College & Research Libraries March 2017 Appendix A. Worksheet for Student Search Assignment Search Strategy Development in a Flipped Library Classroom 393 APPENDIX B Rubric for Student Search Assignment Worksheet Section Point Value Examples Notes 3a. Distinct Concepts • 1 point for each relevant distinct concept • Up to 3 points total 1. Soft Drinks 2. Obesity 3. Teens Concepts do not need to be these exact terms, but should be related. Students may also develop more than 3 distinct concepts. Marginal concepts can be assigned ½ point. Completely unrelated concepts receive 0 points. 3b. Synonyms • ½ point for each relevant synonym or related concept • Up to 4 points total 1. soda, carbonated beverage, sugar- sweetened beverage, cola 2. obese, overweight, BMI, body mass 3. teenager, juvenile, youth, adolescent, children Synonyms or related concepts should be relevant to the research questions, though there may be great variation in terminology. Irrelevant terms receive 0 points. 4a. Final Search Statement • 1 point for appropriate use of AND operator • 1 point for appropriate use of OR operator • ½ point for each relevant concept and/or synonym, up to 5 points • ½ point for appropriate use of truncation • ½ point for appropriate use of quotations • Up to 8 points total ("soft drink*" OR soda OR carbonated OR sugar* beverage* OR cola) AND (obesity OR obese OR overweight OR BMI OR “body mass”) AND (teen* OR juvenile* OR youth OR adolescen*) If truncated term likely to retrieve unwanted results (such as gene* = gene, genetics, but also general, generation), do not assign that term any points. 4b. Limiters/ Modifiers • ½ point for each appropriate use of limiter (such as title search, narrow by date range, research articles) • Up to 2 points total Title search = “soft drink*” and obesity fields Date range = 2000 to 2014 Document types = articles These limiters may be listed under 4a and should also be given credit for this section 4c. Total Number of Articles in Final Search • Assign 0 points if: = 0 or ≥ 500 • Assign 1 point if: (> 0 and < 5) or (≥ 150 and < 500) • Assign 2 points if: (≥ 5 and < 10) or (> 75 and < 150) • Assign 3 points if: (≥ 10 and ≤ 75) 394 College & Research Libraries March 2017 Appendix C. Student Session Evaluation Form Notes 1. Dan Berrett, “How ‘Flipping’ the Classroom Can Improve the Traditional Lecture,” Educa- tion Digest 78, no. 1 (2012): 36–41. 2. Jonathan Bergmann and Aaron Sams, “Remixing Chemistry Class: Two Colorado Teachers Make Vodcasts of Their Lectures to Free Up Class Time for Hands-on Activities,” Learning & Leading with Technology 36, no. 4 (2009): 22–27. 3. Penny M. Beile and David N. Boote, “Does the Medium Matter? A Comparison of a Web- Based Tutorial with Face-to-Face Library Instruction on Education Students’ Self-Efficacy Levels and Learning Outcomes,” Research Strategies 20, no. 1/2 (2005): 57–68; Carol Anne Germain, Trudi E. Jacobson, and Sue A. Kaczor, “A Comparison of the Effectiveness of Presentation Formats for Instruction: Teaching First-Year Students,” College & Research Libraries 61, no. 1 (2000): 65–72; James Nichols, Barbara Shaffer, and Karen Shockey, “Changing the Face of Instruction: Is Online or In- Class More Effective?” College & Research Libraries 64, no. 5 (2003): 378–88; Laura M. Schimming, “Measuring Medical Student Preference: A Comparison of Classroom Versus Online Instruction for Teaching PubMed,” Journal of the Medical Library Association 96, no. 3 (2008): 217–22; Barbara A. Shaffer, “Graduate Student Library Research Skills: Is Online Instruction Effective?” Journal of Library & Information Services in Distance Learning 5, no. 1/2 (2011): 35–55; Brighid M. Gonzales, “Online Tutorials and Effective Information Literacy Instruction for Distance Learners,” Journal of Library & Information Services in Distance Learning 8, no. 1/2 (2014): 45–55; Li Zhang, Erin M. Watson, and Laura Banfield, “The Efficacy of Computer-Assisted Instruction Versus Face-to-Face Instruction in Academic Libraries: A Systematic Review,” Journal of Academic Librarianship 33, no. 4 (2007): 478–84. 4. Diane Fulkerson, “The Flipped Classroom and Media for Library Instruction: Changing Library Instruction,” Against the Grain 26, no. 4 (2014): 17–21. 5. Seth Allen, “Shifting the Instructional Paradigm: Articulating a Set of Current Practices in Flipped Library Instruction,” Tennessee Libraries 64, no. 3 (2014): 1; Sara Arnold-Garza, “The Flipped Classroom Teaching Model and Its Use for Information Literacy Instruction,” Communications in Information Literacy 8, no. 1 (2014): 7–22; Ilka Datig and Claire Ruswick, “Four Quick Flips: Activities for the Information Literacy Classroom,” College & Research Libraries News 74, no. 5 (2013): 249–57; Catherine A. Lemmer, “A View from the Flip Side: Using the ‘Inverted Classroom’ to Enhance the Legal Information Literacy of the International Ll.M. Student,” Law Library Journal 105, no. 4 (2013): 461–91; C. Andrew Youngkin, “The Flipped Classroom: Practices and Opportunities for Health Sciences Librarians,” Medical Reference Services Quarterly 33, no. 4 (2014): 367–74; Elizabeth A. Gibes and Heather James, “Is Flipping Enough? A Mixed Approach to Introductory Information Literacy Instruction,” College & Research Libraries News 76, no. 1 (2015): 10–13; M. Leslie Madden and Ida T. Martinez, “The Flipped Library Classroom at Georgia State University: A Case Study,” Search Strategy Development in a Flipped Library Classroom 395 Georgia Library Quarterly 52, no. 1 (2015): 13–20; Tasha Maddison, Donna Beneteau, and Brandy Sokoloski, “Breaking Ground: Improving Undergraduate Engineering Projects through Flipped Teaching of Literature Search Techniques,” Issues in Science and Technology Librarianship 78 (2014): 2. 6. Christopher J. Mortensen and Angie M. Nicholson, “The Flipped Classroom Stimulates Greater Learning and Is a Modern 21st Century Approach to Teaching Today’s Undergraduates,” Journal of Animal Science 93, no. 7 (2015): 3722–31. 7. Kathy Missildine, Rebecca Fountain, Lynn Summers, and Kevin Gosselin, “Flipping the Classroom to Improve Student Performance and Satisfaction,” Journal of Nursing Education 52, no. 10 (2013): 597–99. 8. Cheryl P. Talley and Stephen Scherer, “The Enhanced Flipped Classroom: Increasing Aca- demic Performance with Student-Recorded Lectures and Practice Testing in a ‘Flipped’ Stem Course,” Journal of Negro Education 82, no. 3 (2013): 339–47. 9. Jack F. Eichler and Junelyn Peeples, “Flipped Classroom Modules for Large Enrollment General Chemistry Courses: A Low Barrier Approach to Increase Active Learning and Improve Student Grades,” Chemistry Education Research and Practice 17, no. 1 (2016): 197–208. 10. Jacqueline O’Flaherty and Craig Phillips, “The Use of Flipped Classrooms in Higher Edu- cation: A Scoping Review,” Internet and Higher Education 25 (2015): 85–95. 11. Jamie L. Jensen, Tyler A. Kummer, and Patricia D. Godoy, “Improvements from a Flipped Classroom May Simply Be the Fruits of Active Learning,” CBE Life Sciences Education 14, no. 1 (2015): ar5. 12. Susan L. Hotle and Laurie A. Garrow, “Effects of the Traditional and Flipped Classrooms on Undergraduate Student Opinions and Success,” Journal of Professional Issues in Engineering Education & Practice 142, no. 1 (2016): 05015005. 13. Randall S. Davies, Douglas L. Dean, and Nick Ball, “Flipping the Classroom and Instruc- tional Technology Integration in a College-Level Information Systems Spreadsheet Course,” Educational Technology Research & Development 61, no. 4 (2013): 563–80. 14. Karen Anderson and Frances A. May, “Does the Method of Instruction Matter? An Experi- mental Examination of Information Literacy Instruction in the Online, Blended, and Face-to-Face Classrooms,” Journal of Academic Librarianship 36, no. 6 (2010): 495–500. 15. Andrea W. Brooks, “Information Literacy and the Flipped Classroom: Examining the Impact of a One-Shot Flipped Class on Student Learning and Perceptions,” Communications in Information Literacy 8, no. 2 (2014): 225–35. 16. Eduardo Rivera, “Using the Flipped Classroom Model in Your Library Instruction Course,” Reference Librarian 56, no. 1 (2015): 34–41.