VanScoy.indd 566 Evidence vs. Anecdote: Using Syllabi to Plan Curriculum-Integrated Information Literacy Instruction Amy VanScoy and Megan J. Oakleaf Amy VanScoy is Associate Head, Research and Information Services at North Carolina State University; e-mail: amy_vanscoy@ncsu.edu. Megan J. Oakleaf is Assistant Professor, School of Information Studies, at Syracuse University; moakleaf@syr.edu. Many academic library instruction programs seek to integrate informa- tion literacy skills into the curriculum of academic departments. Previ- ous literature on this topic generally recommends a “tiered” approach to curriculum-integrated instruction (CII); these tiered approaches suggest teaching basic skills to first- and second-year students and advanced skills to third- and fourth-year students. Many authors identify skills to teach at each level; however, their recommendations appear to be based on anecdote and common sense. While both anecdote and common sense are useful as starting points, librarians who plan CII programs should use evidence to make instructional decisions. To provide evidence for CII planning, this syllabus study investigated the research skills required of first-year students in their first semester at college. The results demon- strate that most first-semester students are required to find articles and Web sites to support their assignments, and many students are required to find books. Some must also find reference books as well as data and statistical sources to complete their course assignments. These results suggest that previous recommendations regarding tiered instructional approaches should be investigated further and revised. ncreasingly, academic librar- ies seek to integrate informa- tion literacy instruction into the curriculum of academic departments within the university. The growth of CII programs is visible in the library and information sciences lit- erature. The literature also reveals that many librarians recommend a “tiered” approach to CII in which some skills are taught at an early level and other skills are taught later in a student’s college career. In general, these recommendations focus on instructional efforts on individual campuses. However, two flaws exist with these suggestions. First, they do not ap- pear to be based on research. While years of professional experience give individual librarians a wealth of anecdotal data from which to conceive CII programs, academic librarians should conduct research to con- firm their assumptions. Second, most CII literature focuses on majors or disciplines. This perspective is conceptually useful for librarians, but students’ academic experiences do not fall neatly within Using Syllabi to Plan Curriculum-Integrated Information Literacy Instruction 567 the boundaries of a particular major or discipline. Students take a wide variety of courses, especially at the beginning of their academic careers. Therefore, the fo- cus of CII research should be approached from a student perspective, not a major or discipline perspective. This article describes a student-cen- tered, evidence-based approach to plan- ning a CII program at a large, research- extensive institution in the Southeast. The study focuses on first-year students in their first semester at college. The central research question investigates the research skills that first-semester, first- year students are required to perform. To answer this question, a random sample of students was selected and their course syllabi were collected and analyzed. The results of this study can be used to determine which skills to include in the most basic CII tier; they also can be used to compare previous CII models with research evidence. Literature Review Curriculum-Integrated Instruction The goal of CII is “to sequence IL through- out the general education and upper division curricula in ways that better meet students’ needs as they advance in their coursework.”1 Library and in- formation science literature reveals that CII programs tend to have multiple tiers ranging from basic to advanced. There is no consensus on which skills are “basic” and which ones are “advanced,” and there are no recommendations on how to distinguish the two. In fact, many authors do not explicitly state the basis for their recommendations; most appear to rely on common sense or professional experience. Most CII literature focuses on ways to integrate basic and advanced information literacy skills into a particular discipline. For example, several authors describe the use of CII in medical education. Du- rando and Oakley advocate the use of CII in nursing education, but they do not identify which skills should be taught at which levels.2 A study by Moore et al also describes a tiered approach to instruction for nursing students, but they do not pro- vide detailed information about the skills. For example, they say only that “selected outcomes” from the Objectives for Informa- tion Literacy Instruction: A Model Statement for Academic Librarians should be taught at the basic level.3 A third description of CII in nursing education is provided by Dorner, Taylor, and Hodson-Carlton, who suggest a list of skills to teach nursing students, but—like Durando and Oakley and Moore et al—they do not reveal the process by which they selected the skills.4 Haraldstad describes a CII program for medical students in Norway. She suggests teaching a number of skills to first-year medical students.5 These skills are derived partially from informal feedback from students6 and partially from other librar- ians’ suggestions.7 A number of authors recommend CII programs for students in other scientific fields and the social sciences. For example, Nerz and Weiner developed a four-year CII program for students in engineering.8 The skills addressed by their program are drawn from Lin’s proposed competencies for professional engineers.9 Brown and Krumholz present a model for integrating information literacy into geomicrobiology courses, but focus only on upper-level research skills.10 Smith lists specific skills for the sciences and recommends a two- tiered approach—one for freshmen and sophomores, another for juniors and seniors.11 Carr and Somerville propose a two-tiered program of skills for chemistry students.12 None of these authors articu- late the criteria used to assign skills to tiers.13 In the social sciences, one model for CII exists. Affleck describes a CII program for education students. Affleck generated a list of skills based on faculty surveys but does not separate the skills into basic and advanced levels.14 Although the discipline-focused CII literature provides interesting insight into what librarians believe students studying specific subject areas should know, it is 568 College & Research Libraries November 2008 important to note that these recommen- dations have one important limitation: they view students’ learning needs from a librarian’s, rather than a student’s, per- spective. College students do not enroll exclusively in courses in their major area. In fact, the early semesters of a col- lege career are dominated by nonmajor classes. As a result, librarians who plan CII programs must consider students’ needs across the curriculum. While most CII literature is discipline- focused, there are a few exceptions. Ury suggests a tiered approach that is not linked to a particular curriculum, but she does not explain how she assigned skills to each tier.15 Orr, Appleton, and Wallin de- scribe a complex “conceptual framework” for tiered IL instruction; however, they do not include skills to be taught at each level of the framework.16 MacDonald et al advocate a tiered approach to information literacy instruction and list specific skills to be addressed in a for-credit course.17 How- ever, they do not indicate when students should take the course or how the course fits in with other levels of instruction. Har- rison and Rourke recommend integrating information literacy skills into the general college curriculum, but they do not list specific skills to include.18 Holliday and Fagerheim also present a non–subject-fo- cused CII approach, although they embed the instruction in English composition courses.19 Unlike other authors who do not reveal the origin of their recommendations, Holliday and Fagerheim used a review of the literature, a survey of librarians, and a survey of English instructors to generate a list of skills and phrased them accord- ing to the Information Literacy Competency Standards for Higher Education.20 In sum, many authors recommend using tiered CII programs to teach infor- mation literacy skills and have contrib- uted significant effort in planning these programs. Several have produced lists of skills that should be taught at a basic or advanced level. A few have justified the recommendations for placing a specific information literacy skill in either the ba- sic or advanced tier. Unfortunately, almost none have revealed the evidence for their recommendations. As a result, librarians who seek to plan evidence-based CII pro- grams must conduct research of their own to determine what skills students need at each level of their college careers. Syllabus Studies Because the literature provides insufficient evidence on which to base CII programs, librarians must undertake research to ensure that students learn what they need to complete course assignments. Syllabus studies are a useful methodology for investigating the research tasks required of students; they provide realistic and complete pictures of students’ needs. Syl- labus studies have been undertaken by past researchers. According to Rambler, “decisions and actions based at least in part on findings from a syllabus study can facilitate the creation of the ideally responsive and completely curriculum- integrated library.”21 Sayles also advocates the study of syllabi. He states, “If it is well wri�en and complete, it [the syllabus] can provide the librarian with enough infor- mation” to plan for instruction.22 Dewald acknowledges both the time required for syllabus studies and their benefits. She writes, “While a syllabus study can be time consuming, it can provide valuable information for librarians in improving academic library services.”23 Many early syllabus studies focus on how much students are required to use the library, rather than examining the informa- tion literacy skills they need to accomplish their course assignments. Rambler studied 162 randomly sampled syllabi for under- graduate courses and found that 73%–77% of lower-level courses required no library research.24 Lauer, Merz, and Craig also examined syllabi to determine the amount of research-based library use and the sophistication of this use by course level. As Rambler did, Lauer, Merz, and Craig found that lower-level courses required less intensive library use.25 More recent syllabus studies reveal increased use of Using Syllabi to Plan Curriculum-Integrated Information Literacy Instruction 569 libraries. Dewald studied the amount of “library use or required research,”26 as well as “the types of assignments given to business students”;27 however, she limited her study to courses required of business majors and did not include electives or multiple sections of courses. Dewald found that 56%–60% of 100-level courses required “no research or library use”; 51%–56% required “some research or library use for shorter assignments”; and only 8%–9% required “significant research projects.”28 Dewald also recognized the need to consider required courses outside the business school. She estimated that 94 percent of business students would have to do some research during their first year as a result of English composition and a first-year seminar.29 Other syllabus studies focus on types of assignments requiring library use. For example, Rambler completed a syllabus study that categorized library assign- ments into groups including “library reviews,” “periodical assignments,” and “research.”30 Bean and Klekowski exam- ined syllabi for categories of library use including “research paper/report/project” and “bibliography or literature search.” Bean and Klekowski also included an “other” category made up of “journal,” “essay,” and “case study” assignments.31 Williams, Cody, and Parnell mined online syllabi for types of library assignments such as “research papers/projects/reports/ speeches,” “group projects, books review and special projects,” and other categories for reserve materials and optional use of library materials.32 These syllabus studies provide help- ful insight into how much students use libraries and what types of assignments they are asked to complete. However, they do not focus on the outcomes students must achieve to be successful; consequently, additional investigation is merited in this area. Methodology T h i s s t u d y e m p l o y e d a s y l l a b u s methodology to examine the research tasks required of first-semester, first- year students at a large university in the southeastern United States that admits approximately 4,000 new students per year. A random sample of 350 first-semes- ter, first-year students was obtained from the university registrar, a sample size large enough to make inferences to the population of first-year students at the university. The information provided for each student included the sections and courses in which the student was enrolled. This information was entered into a relational database for analysis. Then, the syllabi and available assign- ment descriptions for each student were collected via course Web sites or through instructor contacts. Because not all instructors responded to the research- ers’ request for information, syllabi for every course could not be obtained. In fact, complete course information was collected for only 139 students. Next, the course information for all 350 students was carefully examined. If class assign- ments did not require students to com- plete any research tasks, a value of “no” was entered into the database. When class assignments required completion of a research task, a value of “yes” was entered and the assignments were ana- lyzed to determine which broad types of research tasks were required. Because CII programs are skill-based approaches to instruction, librarians who develop them need to know what skills students are required to demon- strate to complete their assignments. As a result, the categories used in this study focus on the outcomes students are required to demonstrate through the completion of tasks, rather than the amount of library use or the assignment outputs (such as a “research paper” or “annotated bibliography”). The broad categories of outcomes were: 1) find Web sites; 2) find articles; 3) find books; 4) find reference books; and 5) find data and statistics. At first glance, these out- come categories appear to be focused 570 College & Research Libraries November 2008 on Standard 2 of the Information Literacy Competency Standards for Higher Educa- tion, “The information literate student accesses needed information effectively and efficiently.” However, each outcome category includes skills associated with other Standards. For example, students required to “find articles” must be able to recognize the need for an article, select a database, construct a search, evaluate results, locate a print or online copy, and cite the article—skills that can be mapped to multiple Standards. Results For the total sample of 350 students, anal- ysis of student syllabi and assignment de- scriptions revealed that 97 percent of the students were required to find research resources during the first semester of their first year of college. For the 139 students for whom complete course information was collected, all were required to find research resources. When the data was examined by type of resource, 87 percent of the 350 students studied were required to find articles to FIGURE 1 Students Required to Find Specific Resource Types Based on Partial Course Information (n = 350) 87 84 74 29 27 0 10 20 30 40 50 60 70 80 90 100 Articles Web sites Books Reference Books Data & Statistics % FIGURE 2 Students Required to Find Specific Resource Types Based on Complete Course Information (n = 139) 94 95 85 40 40 0 10 20 30 40 50 60 70 80 90 100 Articles Web sites Books Reference Books Data & Statistics % Using Syllabi to Plan Curriculum-Integrated Information Literacy Instruction 571 complete their assignments, 84 percent were required to find Web sites, and 74 percent were required to find books (see figure 1). Fewer students were required to find reference books (29%) and data or statistics (27%) to complete their courses. These percentages increased for the 139 students for whom complete course information was available (see figure 2). Ninety-four percent of students were required to find articles, 95 percent were required to find Web sites, and 85 percent were required to find books. Forty percent of these students were required to find reference books and data or statistics. Data were also examined to deter- mine the variety of types of resources an individual student may have to find. Seventeen percent of the 350 students were required to find all five resource types examined for this study (see figure 3), although, for some assignments, it FIGURE 3 Students Required to Find Multiple Resource Types Based on Partial Course Information (n=350) 91 83 75 34 17 9 0 10 20 30 40 50 60 70 80 90 100 At least one Two or more Three or more Four or more All types Could not be determined % FIGURE 4 Students Required to Find Multiple Resource Types Based on Complete Course Information (n=139) 100 96 89 52 26 0 10 20 30 40 50 60 70 80 90 100 At least one Two or more Three or more Four or more All types % 572 College & Research Libraries November 2008 was not clear exactly which types of re- sources were required. For the group of 139 students for whom complete course information was collected, 26 percent were required to find all research types (see figure 4). The finding that 97 percent of first- year students are required to find re- search resources in their first semester of college confirms Dewald’s estimation that 94 percent of first-year students are required to locate research resources, an estimation based on the research demands of first-year writing courses.33 However, not all institutions require a writing course in the first semester, as is the case at the study institution. To determine the impact of first-year writ- ing courses on the study results, the data was analyzed again, using course information from only the 197 students of the sample that were not enrolled in the first-year writing course. This analy- sis revealed that, among the students not enrolled in first-year writing, 82 percent were required to find articles, 83 percent were required to find Web sites, and 74 percent were required to find books. In addition, 30 percent were required to locate reference books, and 28 percent were required to locate data and statistics. Discussion The study results suggest that many early recommendations regarding tiered instructional approaches should be re- examined. Indeed, the evidence generated by this study can be directly compared to several previous works. For example, the current study revealed that 91 percent of first-year students in the study sample were required to find a least one resource type. This result updates conclusions that 56%–77% of lower-level courses don’t re- quire research.34 The earlier findings sug- gest that first-semester, first-year students are not required to complete many re- search tasks. However, a student-centered examination of syllabi and assignment descriptions reveals a different picture. This study suggests that that most first- semester students need to find articles, Web sites, and books to complete their assignments. This may not be a surprise to many librarians; however, some of the proposed CII programs do not cover these tasks in the basic tier. For example, a few authors list finding articles as an important first-year skill,35 but others suggest waiting until the sophomore year,36 a recommen- dation that is not supported by the finding that 87 percent of students must achieve this outcome in their first semester of col- lege. Furthermore, although the current FIGURE 5 Students Not Enrolled in First-Year Writing Who Were Required to Find Specific Resource Types (n=197) 82 83 74 30 28 0 10 20 30 40 50 60 70 80 90 100 Articles Web sites Books Reference Books Data & Statistics % Using Syllabi to Plan Curriculum-Integrated Information Literacy Instruction 573 study finds that 84 percent of incoming students must find Web sites to complete their assignments, only Smith and Harald- stad list the skill at a beginning level.37 Ury and Dorner, Taylor and Hodson-Carlton list finding Web sites as an upper-level skill,38 while Nerz and Weiner and Carr and Somerville omit this skill from their CII programs.39 The current study also showed that 74 percent of students are required to find books. Fortunately, most of CII programs suggest that this skill be addressed during the freshman year.40 Ury recommends it for the sophomore year,41 and Carr and Somerville do not include this skill at all.42 This study also shows that nearly a third of incoming students are required to find reference books to complete their course assignments. Several authors recommend teaching students this skill during the basic level of CII.43 On the other hand, Ury recommends finding reference books as an upper-level skill,44 and Dorner, Taylor, and Hodson-Carlton and Haraldstad do not include this skill in their CII recommendations.45 Like reference books, more than a quarter of students must find data and statistics to complete their assignments. However, none of the previous CII literature sug- gests teaching students this skill. Recommendations for Future Research Much of the previous CII literature fo- cuses on students in individual majors or disciplines. To gain a complete picture of the research skills required of first-year students, the present study did not adopt this approach. However, at some institu- tions, students enroll in courses in their major in their first semester and pursue a strictly discipline-focused track. For these institutions, it is possible that there would be some disciplinary differences between students in various majors. In this study, the sample size was not large enough to allow analysis by major. Still, a discipline-focused approach to the CII research merits future investigation. Conclusion This study demonstrates that first-se- mester, first-year students have required research needs: they must be able to find articles, Web sites, and books to complete their assignments. For some incoming students, all of these skills are mandatory. Therefore, the basic tier of CII should in- clude, at a minimum, instruction on find- ing articles, Web sites, and books. Nearly a third of students require additional skills to successfully complete assignments in their first semester, including finding ref- erence books and data or statistics. It is important to note that this re- search, conducted during one semester at one institution, indicates the presence of trends in the needs that first-year students have, but it is not necessarily generalizable to other colleges and uni- versities. As a result, librarians planning CII programs should undertake their own syllabus studies. The evidence pro- duced by future studies could be used locally to inform instructional decision- making and be shared more broadly to continue the professional conversation on this topic. Once constructed, an evi- dence base for CII will allow librarians to develop sound, strategic instruction and help students achieve the skills they need to succeed. Notes 1. Wendy Holliday and Bri� Fagerheim, “Integrating Information Literacy with a Sequenced English Composition Curriculum,” portal: Libraries and the Academy 6 (Apr. 2006): 170. Available online at h�p://muse.jhu.edu/journals/portal_libraries_and_the_academy/v006/6.2holliday.pdf. [Accessed 30 September 2008]. 2. Paola Durando and Patricia Oakley, “Developing Information Literacy Skills in Nursing and Rehabilitation Therapy Students,” Journal of the Canadian Health Libraries Association 26 (Winter 2005): 7–11. 574 College & Research Libraries November 2008 3. Deborah Moore, Steve Brewster, Cynthia Dorroh, and Michael Moreau, “Information Competency Instruction in a Two-Year College: One Size Does Not Fit All,” Reference Services Review 30, no. 4 (2002): 301. 4. Jennifer L. Dorner, Susan E. Taylor, and Kay Hodson-Carlton, “Faculty-Librarian Col- laboration for Nursing Information Literacy: A Tiered Approach,” Reference Services Review 29, no. 2 (2001): 136. 5. Anne-Marie Haraldstad, “Information Literacy: Curriculum Integration with Medical School’s Syllabus,” Liber Quarterly: The Journal of European Research Libraries 12, no. 2/3 (2002): 195. 6. Ibid., 193. 7. Ibid., 194. 8. Honora F. Nerz and Suzanne T. Weiner, “Information Competencies: A Strategic Approach” (paper presented at the annual conference and exposition of the American Society for Engineering Education, Albuquerque, N.M., June 24–27, 2001). 9. Poping Lin, “Leading Ideas: Core Information Competencies Redefined: A Study of The Information Education of Engineers,” Association of Research Libraries. Available online at www. arl.org/bm~doc/li11.pdf. [Accessed 30 September 2008]. 10. Cecelia Brown and Lee R. Krumholz, “Integrating Information Literacy into the Sciences Curriculum,” College & Research Libraries 63 (Mar. 2002): 111–23. 11. Eleanor M. Smith, “Developing an Information Skills Curriculum for the Sciences,” Issues in Science and Technology Librarianship no. 37 (Spring 2003). Available online at www.istl.org/03- spring/article8.html. [Accessed 30 September 2008]. 12. Carol Carr and Arleen Somerville, “Coping with the Transformation of Chemical Informa- tion,” in Using Computers in Chemistry and Chemical Education, ed. Theresa J. Zielinski and Mary L. Swi� (Washington, D.C.: ACS, 1997), 109–31. 13. Brown and Krumholz, “Integrating Information Literacy into the Sciences Curriculum”; Carr and Somerville, “Coping with the Transformation of Chemical Information”; Smith, “De- veloping an Information Skills Curriculum for the Sciences.” 14. Del Affleck, “A Curriculum-Integrated Bibliographic Instruction Programme at the Aca- demic Level,” Education Libraries Bulletin 29 (Summer 1986): 43–47. 15. Connie Ury, “A Tiered Approach to Bibliographic Instruction: The MEDAL Program,” Research Strategies 12 (Fall 1994): 247. 16. Debbie Orr, Margaret Appleton, and Margie Wallin, “Information Literacy and Flexible Delivery: Creating a Conceptual Framework and Model,” Journal of Academic Librarianship 27 (Nov. 2001): 457–63. 17. Mary C. MacDonald, Andree J. Rathemacher, and Joanna M. Burkhardt, “Challenges in Building an Incremental, Multi-Year Information Literacy Plan,” Reference Services Review 28, no. 3 (2000): 240–47. 18. Justin Harrison and Lorna Rourke, “The Benefits of Buy-In: Integrating Information Literacy into Each Year of an Academic Program,” Reference Services Review 34, no. 4 (2006): 599–606. 19. Holliday and Fagerheim, “Integrating Information Literacy with a Sequenced English Composition Curriculum.” 20. Ibid., 179. 21. Linda K. Rambler, “Syllabus Study: Key to a Responsive Academic Library,” Journal of Academic Librarianship 8 (Jul. 1982): 156. 22. Jeremy W. Sayles, “Course Information Analysis: Foundation for Creative Library Sup- port,” Journal of Academic Librarianship 10 (Jan. 1985): 344. 23. Nancy H. Dewald, “Anticipating Library Use by Business Students: The Uses of a Syllabus Study,” Research Strategies 19, no. 1 (2003): 33–34. 24. Rambler, “Syllabus Study,” 159. 25. Jonathan D. Lauer, Lawrie H. Merz, and Susan L. Craig, “What Syllabi Reveal about Library Use: A Comparative Look at Two Private Academic Institutions,” Research Strategies 7 (Fall 1989): 171. 26. Dewald, “Anticipating Library Use by Business Students,” 34. 27. Ibid., 35. 28. Ibid., 37. 29. Ibid., 42. 30. Rambler, “Syllabus Study,” 157. 31. Rick Bean and Lynn M. Klekowski, “Course Syllabi: Extracting Their Hidden Potential” (paper presented at the Sixth Annual Off-Campus Library Services Conference, Kansas City, Mo., Oct. 1993): 3–4. 32. Lisa M. Williams, Sue Ann Cody, and Jerry Parnell, “Prospecting for New Collaborations: Mining Syllabi for Library Service Opportunities,” Journal of Academic Librarianship 30 (Jul. 2004): Using Syllabi to Plan Curriculum-Integrated Information Literacy Instruction 575 270–75. 33. Dewald, “Anticipating Library Use by Business Students,” 42. 34. Dewald, “Anticipating Library Use by Business Students”; Rambler, “Syllabus Study.” 35. Affleck, “A Curriculum-Integrated Bibliographic Instruction Programme”; Carr and Somerville, “Coping with the Transformation of Chemical Information”; Dorner, Taylor, and Hodson-Carlton, “Faculty-Librarian Collaboration”; Haraldstad, “Information Literacy”; Smith, “Developing an Information Skills Curriculum for the Sciences.” 36. Nerz and Weiner, “Information Competencies”; Ury, “A Tiered Approach to Bibliographic Instruction.” 37. Haraldstad, “Information Literacy”; Smith, “Developing an Information Skills Curriculum for the Sciences.” 38. Dorner, Taylor, and Hodson-Carlton, “Faculty-Librarian Collaboration”; Ury, “A Tiered Approach to Bibliographic Instruction.” 39. Nerz and Weiner, “Information Competencies”; Carr and Somerville, “Coping with the Transformation of Chemical Information.” 40. Affleck, “A Curriculum-Integrated Bibliographic Instruction Programme”; Dorner, Taylor, and Hodson-Carlton, “Faculty-Librarian Collaboration”; Nerz and Weiner, “Information Com- petencies”; Smith, “Developing an Information Skills Curriculum for the Sciences.” 41. Ury, “A Tiered Approach to Bibliographic Instruction.” 42. Carr and Somerville, “Coping with the Transformation of Chemical Information.” 43. Affleck, “A Curriculum-Integrated Bibliographic Instruction Programme”; Carr and Somer- ville, “Coping with the Transformation of Chemical Information”; Nerz and Weiner, “Information Competencies”; Smith, “Developing an Information Skills Curriculum for the Sciences.” 44. Ury, “A Tiered Approach to Bibliographic Instruction.” 45. Dorner, Taylor, and Hodson-Carlton, “Faculty-Librarian Collaboration”; Haraldstad, “Information Literacy.” A suite of services offered by the Association of Research Libraries LibQUAL+ Charting Library Service Quality... ® Looking for library service assessment tools? Over 1000 libraries worldwide have used the LibQUAL+ ® Web-based survey Register to learn about LibQUAL+ ® at the ALA Midwinter Conference in Denver Monday, January 26, 2009, 8:30–10:00 a.m. www.libqual.org/events