Evidence Summary

Google Scholar Retrieves Twice as Many Relevant Citations as PubMed and Provides Greater Full-Text Access for Quick, Clinical Nephrology Searches

A Review of:

Shariff, S. Z., Bejaimal, S. A. D., Sontrop, J. M., Iansavichus, A. V., Haynes, R. B., Weir, M. A., & Garg, A. X. (2013). Retrieving clinical evidence: A comparison of PubMed and Google Scholar for quick clinical searches. Journal of Medical Internet Research, 15(8). doi:10.2196/jmir.2624

Reviewed by:

Diana K. Wakimoto

Online Literacy Librarian

California State University, East Bay

Hayward, California, United States of America

Email: diana.wakimoto@csueastbay.edu

Received: 15 Oct. 2013 Accepted: 6 Feb. 2014

2014 Wakimoto. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by‐nc‐sa/2.5/ca/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

Abstract

Objective – To compare recall and precision of results retrieved by searches in PubMed and Google Scholar for clinical nephrology literature.

Design – Survey questionnaire, comparative.

Setting – Canada.

Subjects – Practicing nephrologists with average age of 48 years and who have practiced nephrology for an average of 15 years.

Methods – The researchers identified 100 systematic reviews in renal therapy published between 2001 and 2009. The primary studies cited in the systematic reviews served as the reference standard for relevant articles; 1,574 unique citations were identified and used to measure recall and precision. The researchers created a unique clinical question from each of the objective statements of systematic reviews and sent one question to a random sample of practicing nephrologists to determine the search strings they would use to search for clinical literature; the researchers collected 100 usable responses. Using the search string in both Google Scholar and PubMed, the researchers analyzed the first 40 retrieved results in each for recall of relevant literature and precision. The researchers also analyzed the availability of full-text articles in each database. A pilot study to test the methodology preceded the main study.

Results – Google Scholar’s recall for the first 40 records was 21.9% and PubMed was 10.9%. Each database contained 78% of the relevant literature/reference standard set from the systematic reviews. However, 15% of the articles were in neither database. Precision results were similar (7.6% for Google Scholar and 5.6% for PubMed). Google Scholar had more full-text available at 15% of articles versus 5% for PubMed. Google Scholar and PubMed had similar numbers of relevant articles when all retrieved records were analyzed, but Google Scholar still provided more access to free full-text articles.

Conclusion – Google Scholar provides better recall and provides more access to full-text than PubMed; however, search strings provided by nephrologists used in both databases failed to retrieve 80% of relevant articles. Therefore improving nephrologists’ ability to effectively search could enhance their ability to implement research in practice helping patients. The researchers suggest future studies should be conducted to determine the generalizability of the findings on recall and precision in other medical disciplines.

Commentary

This study adds to the growing body of literature showing multiple professions’ interest in the usefulness of Google Scholar for searching literature as compared to more established databases such as PubMed (Bourbakhsh, Nugent, Wang, Cevik, & Nugent, 2012). As more people – both researchers and practitioners – become aware of, and start to use, Google Scholar, it becomes even more important to analyze Google Scholar’s recall, precision, and access to full-text articles in relation to other databases. This study’s results will be of great interest to librarians who work with clinicians in medical disciplines and for others who may want to replicate the study’s methodology in their own area of practice.

This is a very strong study methodologically, is well-written, and is useful for both researchers and practitioners. It builds on prior research conducted by the research team on search habits and techniques of nephrologists (Shariff et al., 2011); a critique of another study by the researchers in this area appears in an earlier volume of Evidence Based Library and Information Practice (Kelly, 2012). The current study is considered valid using the critical appraisal checklist by Glynn (2006). The methodology is described in a way that can be replicated and the results are clearly reported. The researchers also document the limitations of the study, which could be tested in future studies. The researchers also provide suggestions for future research to expand the generalizability of the findings.

A minor possible weakness of the article is this: for readers to fully understand the survey questionnaire used, and the characteristics reported on the nephrologists’ searching habits, they would need to read the research team’s previous article which more fully analyzes the survey data (Shariff et al., 2011). However, this is quite minor and the researchers still report adequate information on nephrologist survey respondents for the readers to contextualize the results and discussion.

This research article is significant in its rigorous comparison of Google Scholar and PubMed and provides valuable insight for medical librarians, and possibly librarians who work in other fields as well. As more people use Google Scholar for quickly finding research to use in clinical therapy, it is important that librarians understand the strengths and weaknesses of the database and its place in literature searching. As the researchers note that most nephrologists view fewer than 40 search results, it would be interesting to replicate the study given PubMed’s recently introduced relevancy ranking, a feature that may have given Google Scholar the edge in retrieving more relevant results in the first 40 records. The researchers’ call for helping to improve the searching capabilities of clinicians is one area in which librarians can apply this study’s findings to potentially impact patient care.

References

Glynn, L. (2006). A critical appraisal tool for library and information research. Library Hi Tech, 24(3), 387-399. doi:10.1108/07378830610692154

Kelly, K. (2012). Applying the narrow forms of PubMed methods-based and topic-based filters increases nephrologists’ search efficiency. Evidence Based Library and Information Practice, 7(3), 95-97. Retrieved 17 Feb. 2014 from http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/17687/14307

Nourbakhsh, E., Nugent, R., Wang, H., Cevik, C., & Nugent, K. (2012). Medical literature searches: A comparison of PubMed and Google Scholar. Health Information and Libraries Journal, 29, 214-222. doi:10.1111/j.1471-1842.2012.00992.x

Shariff, S. Z., Bejaimal, S. A. D., Sontrop, J. M., Iansavichus, A. V., Weir, M. A., Haynes, R. B., … Garg, A. X. (2011). Searching for medical information online: A survey of Canadian nephrologists. Journal of Nephrology, 24(6), 723-732. doi:10.5301/JN.2011.6373