Tips from the Experts
What Does Bad Information Look Like? Using the CRAAP Test for Evaluating Substandard Resources
Abbey B. Lewis
STEM Learning and Collections Librarian
Norlin Library
University of Colorado Boulder
Boulder, Colorado
Abbey.B.Lewis@colorado.edu
Abstract
The CRAAP Test (Currency, Relevance, Authority, Accuracy, Purpose), is a popular tool among academic librarians for instructing students in resource evaluation. Its applicability to web-based information makes it particularly useful for students as they think about information they encounter both inside and outside of scholastic settings. However, while students may be capable of explaining concepts like bias, authority, and scholarly publications, when it comes to questions of reliability they may have limited experience. For instance, students may have no trouble recognizing high quality resources, but still have difficulty identifying those characteristics that signify an unreliable or substandard resource. In an undergraduate science writing course, the CRAAP Test was introduced to students, who then used the test in a group activity to evaluate a resource that failed key aspects of the test. This activity engages students in how to critically evaluate resources and provides useful feedback for instructors on those evaluative concepts that remain unclear.
Introduction
Resource evaluation is a key component of library instruction at many higher education institutions that seek to enhance students' information literacy skills. The CRAAP Test (Blakeslee 2004), RADAR (Mandalios 2013), and similar strategies provide a framework for teaching concepts that can be applied to both scholarly and popular resources to gauge their appropriateness and credibility. The Association of College & Research Libraries (ACRL) Board further underscored the importance of critical evaluation of resources in its "Framework for Information Literacy for Higher Education" with its in-depth discussion of concepts, such as authority, that information literate learners can recognize and articulate (ACRL 2016).
However, successful identification of quality information can feel a bit like checking off boxes when an academically suitable resource is used for teaching purposes: here are the author's credentials, here is the title of a reputable publication, and here are the research claims along with data and citations from other reputable publications to support those claims, and so on. Are students really learning to think critically about resources when the resources don't have any identifiable flaws?
Class Activity
The CRAAP Test was introduced to students in an undergraduate science writing course during day two of a two-day library instruction session. Students were somewhat familiar with resource evaluation concepts like bias, credibility, and peer review from previous discussions in the class. At the beginning of the activity, the CRAAP Test was presented as a tool for students to assess the quality of information they were considering for use in an academic assignment. Students were then placed in five groups with each group assigned to answer the questions from either the Currency, Relevance, Authority, Accuracy, or Purpose sections of the CRAAP Test and asked to share their observations via Padlet (Figure 1).
Figure 1
The resource that students were asked to evaluate was an article from a blog that dealt with the topic of climate change from a viewpoint of heavy skepticism (CO2 is Life 2017). While the article could pass some aspects of the CRAAP Test (it was posted fairly recently and the topic of the given article could potentially be of interest to a student researching the scientific causes of climate change), it failed the test in others, often spectacularly so. The article showed evidence of political and ideological bias, no authors or credentials were given (although the blog itself claimed to be "the definitive source for exposing the global warming hoax"), and graphs were presented without any indication of where the data came from and with only tangential ties to the accompanying text (Figure 2 and Figure 3). Students were not informed of the article's shortcomings, but simply instructed to use the article to answer the questions in the CRAAP Test.
Figure 2
Figure 3
Observations
While students were overall successful in applying the CRAAP Test, some points of confusion emerged for each group, which in turn led to productive teaching points.
- The group assigned to the Currency questions was confident in their answers overall, but had some difficulty deciding how recent information needed to be.
- The Relevance group initially assumed that the article was meant for a scholarly audience because it contained graphs.
- The Authority group was unsettled because no information on the authors or their affiliations was provided. They believed they were missing something obvious.
- The Accuracy group expressed difficulty in determining if the article had undergone peer review, although they were able to explain the peer review process.
- The group answering the Purpose questions felt that the article had a heavy bias, but needed some assistance in identifying evidence that was indicative of this condition.
Best Practices for This Activity
Prior to conducting this activity, students should be introduced to concepts like peer review, scholarly vs. popular literature, supporting research claims with evidence, forms of bias, and the ways in which authority is constructed and displayed in different types of literature. With this foundation, the activity then allows students to apply these concepts and serves as a type of assessment tool through which instructors can gauge students' understanding of resource evaluation. Some questions in the CRAAP test have ambiguous answers (Would you be comfortable citing this source in your research paper? Is the author qualified to write on the topic?) and students benefit from having one or more instructors circulating during the activity and helping them form more complete responses. Students should also be provided with a hypothetical topic to ensure that all questions within the test can be answered. Finally, students should be given enough time to read the resource in its entirety and examine peripheral information, such as links, URLs, and the larger context of where, how, and why the information was produced.
Conclusion
While it's unlikely that students will encounter information that fails so many aspects of the CRAAP test in a library database, it's also inevitable that they'll come across questionable resources in their personal, academic, or professional lives. Even within the realm of peer-reviewed literature, students still need to be able to think critically about the quality of information they find. The CRAAP Test offers an excellent framework for students to evaluate resources, but applying that framework to a flawed resource can provide students with the ability to recognize problematic information when they see it again. Future directions for this activity could include a structured assessment of student learning, conducting the activity with a different resource evaluation tool, or using a resource with less obvious flaws for upper division or graduate courses.
References
ACRL (Association of College and Research Libraries). 2016. Framework for information literacy for higher education. [Internet]. [Cited 2017 Dec 22]. Available from: http://www.ala.org/acrl/standards/ilframework
Blakeslee, S. 2004. The CRAAP test. LOEX Quarterly. [Internet]. [Cited 2017 Dec 22];31(3):6-7. Available from: http://commons.emich.edu/cgi/viewcontent.cgi?article=1009&context=loexquarterly
CO2 is Life. 2017. Climate "science" on trial; the forensic files: exhibit J [Internet]. [Cited 2017 Dec 22]. Available from: https://co2islife.wordpress.com/2017/01/17/climate-science-on-trial-the-forensic-files-exhibit-j/
Mandalios, J. 2013. RADAR: An approach for helping students evaluate internet sources. Journal of Information Science 39(4):470-478. DOI: 10.1177/0165551513478889
This work is licensed under a Creative Commons Attribution 4.0 International License. 