Evidence Summary

Both Online and In-Person Library Orientations Have Positive Effects on Graduate Students’ Information Literacy Skills

A Review of:

Gall, D. (2014). Facing off: Comparing an in-person library orientation lecture with an asynchronous online library orientation. Journal of Library & Information Services in Distance Learning, 8(3-4), 275-287. doi:10.1080/1533290X.2014.945873

Reviewed by:

Lisa Shen

Head of Reference and Instructional Services

Newton Gresham Library

Sam Houston State University

Huntsville, Texas, United States of America

Email: lshen@shsu.edu

Received: 8 Mar. 2016 Accepted: 20 Apr. 2016

2016 Shen. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

Abstract

Objective – To assess and compare the effectiveness of online and face-to-face library orientations.

Design – Pretest/posttest.

Setting – A public university in the United States of America.

Subjects – Graduate students in a Master of Social Work program.

Methods – At the beginning of the fall 2013 semester, students in 3 different sections of a social work research methods course were asked to complete a 17-question assessment to evaluate their information literacy skills and knowledge. Then, 1 section (Campus A) received a 50-minute in-class library orientation from a librarian, while the other 2 sections (Campus B and Off Campus) received orientation through asynchronous online video tutorials.

Post library orientation, research labs were then held for all three class sections, during which students received some hands-on time working with a librarian. After the hands-on labs, students were invited to complete the posttest assessment, which consisted of the same 17 questions.

Main Results – The researcher collected 59 pre-test and 27 posttest responses, although many respondents did not answer all the questions. Moreover, none of the posttest responses from the Off Campus students was deemed usable by the researcher.

After attending the library orientation and lab sessions, students were more likely to choose the library or a librarian as their starting point for research (19% pretest, 40% posttest). Students’ ability to identify book or chapter title in a citation (48% pretest, 92% posttest), and determine whether common knowledge required citations (87% correct in pre-test, 100% posttest) also appeared to improve after the library sessions. In addition, students’ skills in assessing the scholarliness and credibility of an article by its abstract also improved. While there were some anecdotal variations between responses between Campus A and Campus B groups, no statistically significant differences were noted.

Conclusion – The study results suggest that regardless of format, library orientations and hands on lab session had positive effects on graduate students’ information literacy skills and knowledge.

Commentary

This article strives to make a timely comparison between the effectiveness of online and in-person library instructions, and provides a survey of the related literature. The idea for the study is sound and would provide a good starting point for future research. However, due to flaws in the research design and execution, there are significant concerns with the study’s findings. An examination of this research using the EBL Critical Appraisal Checklist (Glynn, 2006) yielded an overall validity of 32%, below the accepted threshold of 75%.

Nonetheless, the author should be commended for identifying and acknowledging several study limitations, including small sample size, high attrition rate, and errors in the assessment instrument. These observed limitations, such as the lack of proofreading before questionnaire distribution, will help others to minimize similar issues in future research.

On the other hand, readers would have benefited from some additional details from the author. For instance, the rationale and sources used in developing the assessment instrument were not addressed, the content of the in-person orientation and online tutorials were not discussed, and a copy of the actual instrument was unavailable. Consequently, it would be difficult for interested readers to adopt and replicate similar studies.

There were also indications of personal biases in the assessment design and evaluation. In one instance, the author showed preferences for “ask a librarian” and “head to the library” as good choices for the question about starting points for one’s research, but neglected to provide online library resources as an answer option. Likewise, after acknowledging that “there [was] not a correct answer” for a different question about database selection, the author then evaluated the responses based on the choice he had believed to be “the best.” Similarly troubling, none of the posttest responses from the Off Campus group was deemed usable by the author without further clarification.

Moreover, the study’s actual population size, and thus the overall response rate, was unavailable. Because participation was anonymous and voluntary, not only was the 54% (32 of 59) pretest/posttest attrition a concern, but one also cannot confidently conclude that the posttest responses were made by respondents who had completed the pretest. Response rate for the individual questions was also not consistently provided, even though the author noted that most respondents skipped questions.

Therefore, because of all these issues with the research design and analysis, readers are advised to consider the findings from this study with caution. However, with some improvements such as the addition of a coding method to track individual respondents’ pretest/posttest performance, this article can serve as a good starting point for those interested in similar research.

Reference

Glynn, L. (2006). A critical appraisal tool for library and information research. Library Hi Tech, 24(3), 387-399.