Last time, this column focused on librarians conducting research as part of Step 2 – Find the Evidence – in evidence based library and information practice, and what that might mean for the individual and the profession. It’s time to move to Step 3 – Appraise the Evidence. When you’ve looked to the literature and have found some research articles that just might inform your practice, the next step is to look critically at these articles to assess whether or not the research could be used to improve practice or help inform decision making. Critical appraisal is the process of assessing and interpreting evidence by systematically considering its validity, reliability, and applicability for quantitative articles, and its credibility, transferability, dependability, and confirmability for qualitative articles. Critical appraisal is a developable skill that uses explicit methods.

This process may sound daunting, but I would argue that we do this in our everyday lives to some extent. Reading newspapers, watching television, scanning our RSS feeds…we process what we read, see, and learn through our own experiences, and we don’t take everything at face value. Critical thinking is a necessity in a world where “information overload” is not just a catch phrase. There has to be some way to filter what is out there so that parts of it can be useful to the individual and not just so much white noise. This process, with added rigor, is the next step in the practice of EBL.

There are a few criteria that we either implicitly or explicitly use to determine what is worth reading. There’s the interest factor: Are we interested in the topic of the article as outlined by the abstract or the title? Then there are extrinsic factors: Have we heard of the article? Is it published in a peer reviewed journal? Is the author from a reputable institution? Then there are the intrinsic factors, and these are what the critical appraisal process can tease out. These factors “relate to research design and aspects of methodology” (Booth & Brice, 2004, p. 105).

So to that end, let’s look at some term definitions:

· Validity: “the extent to which the results of the research are likely to be free from bias” (Booth & Brice, 2004, p. 105).

· Reliability: “the likelihood that this study could be reproducible, rather than the results just being a ‘fluke’ or chance result” (Booth & Brice, 2004, p. 105)

· Applicability: “the extent to which the results are likely to impact on practice” or be able to effectively inform decision making (Booth & Brice, 2004, p. 105).

A good round of critical appraisal can help to determine the presence (or not) of these terms, and thus the usability of the research article. Qualitative studies are no less relevant to libraries. Trisha Greenhalgh and Rod Taylor have an excellent article entitled “How to read a paper: Papers that go beyond numbers (qualitative research).” In it, they outline the questions to ask when reading a paper based on qualitative analysis.

So luckily, the tools are out there to aid in the critical appraisal process. In addition to the Greenhalgh & Taylor article, there have been several checklists developed to guide the reader through research articles in a systematic way.

· Lindsay Glynn created a useful checklist especially for library and information science. This checklist can be downloaded here: http://ebltoolkit.pbworks.com/f/EBLCriticalAppraisalChecklist.pdf .

· The CriSTAL checklists for various types of studies (appraising a user study: http://www.shef.ac.uk/scharr/eblib/use.htm and appraising an information needs analysis: http://www.shef.ac.uk/scharr/eblib/use.htm).

· ReLIANT is the Readers’ guide to the Literature on Interventions Addressing the Need for education and training: http://eprints.rclis.org/7163/1/RELIANT__final_.pdf

· Tools were developed through the Critical Appraisal Skills Program (CASP) in the UK: http://www.phru.nhs.uk/pages/PHD/resources.htm

It might seem like there are just too many checklists and tools to wade through to get started. However, I suggest that you choose one checklist to start with. Personally, I recommend the one created by Glynn to get you going. Then sit down with the list and the article and work through them both just to get a feel for the process. You might not be able to answer some of the questions on the checklist, which is alright. Just look at the article as objectively as you can, and a useful assessment of the article will start to emerge.

As I mentioned before, critical appraisal is a skill, and it can be honed through practice. There are a couple of things that you can do to increase your skill level. Journal clubs are a good way to practice critical appraisal in a collegial setting. A journal club is a group of individuals who meet regularly to critically evaluate articles in particular practice settings. A journal club was recently started in the Health Sciences Library at the University of Saskatchewan. Health sciences librarians gather to discuss a previously chosen research article in a critical fashion. This process increases our understanding of research methods, keeps us abreast of research in our field, and provides a comfortable atmosphere in which to expand our critical appraisal skills.

As well, the evidence summaries found in each issue of Evidence Based Library and Information Practice are all examples of the critical appraisal process. Koufogiannakis (2006) states that “reading critical appraisals informs us all of the questions we should be asking when we approach a research article and allows us to become more familiar with a critical approach to reading the literature of our field” (p. 81).

Undertaken as a step in the process of evidence based library and information practice, or as a stand alone endeavor, critical appraisal is a worthwhile activity for information professionals. Bringing your own knowledge and experience to the table in combination with research in our field creates a bridge that spans the research/practice gap.

Next time, a look at applying research evidence to practice, and in particular, applicability: What is it? How do you find it?

References

Booth, A. & Brice, A. (2004). Appraising the evidence. In A. Booth & A. Brice (Eds.) Evidence-based practice for information professionals: A handbook (pp. 104-118). London: Facet.

Booth, A. & Brice, A. (n.d.). Appraising a user study. In CriSTAL online. Retrieved from http://www.shef.ac.uk/scharr/eblib/use.htm

Booth, A. & Brice, A. (n.d.). Appraising an information needs analysis. In CriSTAL online. Retrieved 20 Feb. 2010 from http://www.shef.ac.uk/scharr/eblib/needs.htm

Critical Appraisal Skills Program. (2007). Appraisal tools. In Public Health Resource Unit. Retrieved 20 Feb. 2010 from http://www.phru.nhs.uk/pages/PHD/resources.htm

Glynn, L. (2006). A critical appraisal tool for library and information research. Library Hi Tech, 24(3), 387-399.

Greenhalgh, T. & Taylor, R. (1997). How to read a paper: Papers that go beyond numbers (qualitative research). BMJ, 315(7110), 740-743.

Koufogiannakis, D. (2006). Small steps forward through critical appraisal. Evidence Based Library and Information Practice, 1(1), 81-82.

Koufogiannakis, D., Booth, A., & Brettle, A. (2005). ReLIANT: Reader’s guide to the literature on interventions addressing the need for education and training. LIR, 30(94), 44-51.