Review Article

Mixed Methods Research in Library and Information Science: A Methodological Review

Richard Hayman

Associate Professor & Digital Initiatives Librarian

Mount Royal University

Calgary, Alberta, Canada

Email: rhayman@mtroyal.ca

Erika E. Smith

Assistant Professor & Faculty Development Consultant

Academic Development Centre

Mount Royal University

Calgary, Alberta, Canada

Email: eesmith@mtroyal.ca

Received: 12 Sept. 2019 Accepted: 6 Jan. 2020

2020 Hayman and Smith. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

Data Availability: Hayman, R., & Smith, E. E. (2019). Mixed methods in library and information science bibliographic records (2008-2018) [Data set]. Edmonton, Canada: UAL Dataverse, ver. 1.0, https://doi.org/10.7939/dvn/cxuw6a

DOI: 10.18438/eblip29648

Abstract

Objective - To review mixed methods research trends in the field of library and information science (LIS). In particular, we examine the extent to which research about or using mixed methods has been occurring in library and information science over the past decade (2008-2018), and how much of that mixed methods research is done in health contexts.

Methods - We conducted a methodological review and analysis of mixed methods research (MMR) in LIS for published articles indexed in LISTA and Web of Science. After deduplication and verification for inclusion, we coded 417 articles to identify contributions using or about MMR. Given the connections between evidence based practice in health and LIS, we also identified whether articles about or using mixed methods were health-focused.

Results - We found MMR to be a tiny proportion (less than 0.5%) of the overall LIS research literature. In terms of observable trends, while contributions about MMR remain fairly static, there has been an increase in articles using mixed methods. Of the 417 included articles, 373 (89.5%) primarily used mixed methods and 44 (10.5%) were primarily about MMR. Results also demonstrated that health-related research both using and about mixed methods has a strong presence in the LIS literature, with 136 published articles (32.6% of the total).

Conclusion - Confirming findings of prior analyses of research methods in LIS, our methodological review shows current opportunities to adopt and expand the use of mixed methods research processes. Further contributions about mixed methods research, and ideally connecting research and practice in LIS, are needed. Despite the small proportion of MMR in LIS research, there is an observable increase in the number of publications using mixed methods during this timeframe. The LIS research community can promote additional growth by leveraging this momentum around using mixed methods, and look to translate lessons learned about mixed methods research and practice in health contexts to other LIS settings. Recommendations include developing educational opportunities and learning resources that facilitate wider engagement with MMR in LIS contexts.

Introduction

For those interested in evidence based practice (EBP), there is an increasing array of research methods, strategies, and approaches available today that can be leveraged to foster praxis. Various analyses of the literature point to untapped opportunities for researchers and practitioners in the field of library and information science (LIS) to expand the range of research methods and methodologies utilized, including mixed methods approaches (Aytac & Slutsky, 2014; Chu, 2015; Gauchi Risso, 2016; Ullah & Ameen, 2018). In prompting those in LIS to ask “are we there yet?” regarding adoption of mixed methods research, Fidel’s (2008) analysis demonstrated that MMR was not commonly used or discussed in LIS, concluding that increased awareness would be advantageous to the field. We revisit this overarching question regarding whether LIS has been using or discussing mixed methods during the decade following Fidel’s work. As researcher-practitioners who have realized the value of using mixed methods research (MMR) for scholarship and evidence based practice in our own contexts, we see benefits to an evidence based discussion of current trends and the potential value of MMR. With the goal of exploring ways to expand engagement with mixed methods research in LIS contexts in mind, the purpose of this article is to take stock of mixed methods research trends and issues through a broad methodological review of the LIS literature over a ten-year span.

Aims

To support mixed methods practice Plano Clark and Ivankova (2016) argued that there is great value in consulting literature analyses about the status of mixed methods in the context of a particular research community, especially in the form of methodological reviews and discipline-based discussions. With this in mind, we conducted a methodological review and analysis of mixed methods research in LIS published over the past decade (2008-2018) to address the following research question:

RQ1: To what extent is research about or using mixed methods occurring in library and information science?

Additionally, given the established connections between evidence based practice and evidence based medicine, and their intersection in health librarianship, we also explored the following related research question:

RQ2: Over the same decade, what literature about or using MMR in library and information science has occurred within health contexts?

In light of these research questions, our approach specifically sought to capture the breadth of mixed methods research occurring over time and across a considerable, representative dataset. Adopting this broader approach enabled us to compare findings from other LIS research methods analyses, and using a larger sample than if we had focused on a particular subset of journals.

To encourage further development and application of MMR in ways that are clear and relevant for this disciplinary context we outline recommendations connected to LIS practice. Our goal is to promote further consideration of mixed methods research in ways that can beneficially inform new ways of collecting, using, and integrating evidence in LIS contexts.

Defining Mixed Methods Research

There are several definitions of mixed method research, but a common component of most definitions is that researchers must deliberately combine two or more (usually qualitative and quantitative) research methods in a single study to provide the most comprehensive means of addressing the research problem and questions at hand. Recognizing mixed methods as a research process, Creswell (2008) defined MMR thus:

a broad umbrella term encompassing perspectives that see it as a research method of data collection and analysis, a methodology that spans the process of research from philosophical assumptions to interpretations, a philosophy of research, and a set of procedures used within existing research designs such as case studies, experiments, and narrative projects. (p. 2)

Mixing methods increases our ways of viewing issues, providing more evidence than we would using a single method. In their seminal work on MMR, Johnson, Onwuegbuzie, and Turner (2007) argued that MMR was increasingly being understood as a third research paradigm alongside existing qualitative and quantitative research paradigms, providing opportunities “to consider multiple viewpoints, perspectives, positions, and standpoints” (p.113). MMR helps bridge the divide between quantitative and qualitative research (Creswell & Plano-Clark, 2018), and many researchers relate these aspects of MMR to triangulation, a way of cross-validating information from several sources (Gorman & Clayton, 2005; Connaway & Radford, 2017; Wilson, 2014).

Methods

To examine current research trends surrounding mixed methods, we integrated key strategies outlined by MMR experts Plano Clark and Ivankova (2016) for methodological reviews. They reinforce the value of such work for research and practice, acknowledging the “long history of scholars conducting disciplinary-based methodological reviews in the field of mixed methods research” (p. 256). We follow their recommendation to report the procedures used for identifying the sample of published mixed methods research, and analyze specific dimensions and features reported within those publications to provide insights into patterns and trends, such as the prevalence rate of mixed methods. Our methodological review also draws on useful scoping and mapping review techniques (Grant & Booth, 2009) to illustrate issues over the course of a decade through figures and diagrams.

In examining evidence from the literature in ways that are relevant for those in the field, a methodological review should outline strengths and weaknesses and how these may “constrain or open up opportunities for learning” (Elsevier, n.d., p. 4). Huynh, Hatton-Bowers, and Smith (2019) remind us that conducting a methodological review within a disciplinary context helps identify trends and opportunities for using and improving MMR practices. Finally, Onwuegbuzie, Leech, and Collins (2011) noted that a methodological review can be an end in itself, highlighting the benefit of such reviews for informing practice and understanding the topic being explored.

Sources and search strategies

Our search focused on two primary information resources that index research from LIS contexts: Library, Information Science & Technology Abstracts with Full Text (LISTA, from Ebsco), and Web of Science Core Collection (WoS, from Clarivate Analytics). We selected these based on their disciplinary coverage and the fact that both were accessible through our current institutional subscriptions. To be exhaustive with our WoS search we included six main indices from the WoS Core Collection: Science Citation Index Expanded; Social Sciences Citation Index, Arts & Humanities Citation Index; Conference Proceedings Citation Index-Science; Conference Proceedings Citation Index-Social Science & Humanities; and Emerging Sources Citation Index.

To maintain our focus on the use of MMR in current research while also ensuring feasibility and manageability of the project, we restricted all searches to English-language journal articles published from 2008-2018. We identified and used a variety of phrases to describe our primary topic based on our own knowledge of the subject and research being explored. These phrases reflect the popular terminology used extensively in existing MMR literature, and in many cases echoed the language and labels that authors had used in their studies. Test searches allowed us to refine this list, leading to the search strategies outlined below.

Search strategy for LISTA

As a discipline-specific database, LISTA was our starting place to test keywords/phrases and to focus on LIS-related literature.

Search strategy for WoS

As a large, interdisciplinary index, we relied on built-in tools for limiting to only those publications that belong to LIS. Since WoS has a specific subject category for “Information Science & Library Science” we used this for our first search before searching for keywords/phrases.

Together LISTA and WoS revealed 636 results for further analysis. Figure 1 is a high-level illustration of our process starting from the point when these results were combined, deduplicated, and then checked against include/exclude criteria. Only the final 417 included articles were subsequently coded.

Table 1

Searches Conducted within LISTA

Search	Terms and limits	Results
S1	(No keyword/phrase used to find all results) Limiters: ● Publication date: 2008-01-01 to 2018-12-31 ● Publication type: Academic journal ● Document type: Article ● Language: English	98,343
S2	(DE "Mixed methods research") OR "mixed methods research" OR mmr OR "mixed methodology" OR "mixed research" OR "mixed methods sampling" OR "mixed design" OR "mixed method design" OR "combined methods" OR "mixed methods approach" OR "mixed methods study"	504
S3	S1 AND S2	354

Table 2

Searches Conducted within WoS

Search	Terms and limits	Results
S1	(No keyword/phrase used to find all results) WC=(Information Science & Library Science) AND LANGUAGE: (English) AND DOCUMENT TYPES: (Article) Indexes=SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, ESCI Timespan=2008-2018	51,873
S2	TS="mixed methods research" OR ALL="mixed methods research" OR ALL=mmr OR ALL="mixed methodology" OR ALL="mixed research" OR ALL="mixed methods sampling" OR ALL="mixed design" OR ALL="mixed method design" OR ALL="combined methods" OR ALL="mixed methods approach" OR ALL="mixed methods study"	18,518
S3	S1 AND S2	282

Figure 1

Process for methodological review starting with records captured from LISTA and WoS.

Deduplication

We imported the 636 citations into citation management software Zotero (https://www.zotero.org/), which includes a built-in deduplication function that compares several metadata fields and flags suspected matches. We reviewed each flagged match before removing items that were duplicates, then reviewed the full list again to manually remove additional duplicates that were not flagged as part of the automatic deduplication. The remaining 473 items were sent to the include/exclude process.

Inclusion and exclusion criteria

Both authors reviewed the 473 potential articles remaining after deduplication to determine whether they met our inclusion criteria. We examined each study for the following:

● a research article published in a journal;

● situated in library or information science contexts, as determined by the subject matter or source publication; and

● evidence that the study involved MMR processes, whether

○ reporting on an original study using MMR; or

○ discussing MMR as part of a larger methodological discussion; or

○ a protocol study wherein MMR was part of the proposal and the MMR process was evident.

While titles and abstracts typically served as primary sources of information to determine the MMR processes involved, in several cases these provided insufficient evidence that the study was in fact MMR-based. In such cases we then examined the full text, focusing on the methods section, which proved a reliable way to determine each study’s MMR status. We used traditional subscription databases, open access resources, and third-party tools (e.g., ResearchGate) to find full-text versions.

In the rare circumstance where we were unable to locate full text, we decided to err on the side of caution. In these very few cases we based our decision to include or exclude using the available abstract in tandem with their peer-reviewed status. If the abstract described these works as mixed-methods, and reviewers and journal editors had deemed them fit to be published as such, then we would include these few publications in our sample.

While screening articles for inclusion or exclusion, we identified several articles where authors indicated their study used MMR, but upon reading the article it was clear that they reported on only a single phase or method. For example, we found several studies using a survey or questionnaire with closed- and open-ended questions that described themselves as mixed methods. However, Creswell and Hirose (2019) mark the distinction between survey methodologies, which can include open- and close-ended questions, and mixed methods research proper, which may involve a survey or questionnaire but ultimately requires a combination and integration of multiple research approaches. Based on this definition, we excluded survey-only MMR reports from our dataset. Similarly, since intentionally mixing methods is an essential characteristic of MMR, we excluded studies that merely reported on a single stage of a larger MMR project (e.g., only reporting the qualitative or quantitative phase) when they did not situate or report that data within the wider context, methods, and findings of the rest of the MMR study.

We also excluded obvious false hits, such as a few articles that used our MMR acronym keyword for something other than mixed methods research (e.g., articles discussing vaccines for measles, mumps, and rubella). Since published research articles were our focus, we removed results that had been tagged as articles in their source database but were merely conference abstracts or grey literature reports. The include/exclude process resulted in 417 articles that were sent for coding.

Coding

Since our approach specifically sought to capture the breadth of mixed methods occurring in this dataset, both researchers were in agreement that coding of the remaining articles should be sufficiently high-level in order to support feasibility of this wide scope of research. We aimed to generate a general picture and position of MMR in LIS research over the last decade, rather than focus on the specifics of how MMR manifests. Both authors reviewed the 417 included articles and independently coded each according to whether it was a study that used MMR or whether it was about or discussing MMR. Within these two main categories we also identified those that involved medical- or health-related research. Both researchers reviewed and discussed these categorizations to ensure consensus.

Results

Publication Sources

We briefly explored the source publications for these 417 articles. Concerning RQ1, MMR articles appeared in 121 different publications representing the breadth and depth of LIS research over the past decade. The top five publications and the number of articles from each are in Table 3.

Table 3

Top Five Publication Sources by Number of MMR Articles Published

Publication title	No. of articles
Journal of Medical Internet Research	62
Qualitative Health Research	20
Information Research	10
Journal of the American Medical Informatics Association	10
Journal of the Association for Information Science & Technology (formerly Journal of the American Society for Information Science & Technology)	10

We accounted for identifiable journal title changes that occurred during the decade in question and standardized for slight differences in journal titles within citation information (e.g., Evidence Based Library and Information Practice vs. Evidence Based Library & Information Practice). A full list of publication titles and article counts is available as Appendix A.

Prevalence of MMR

Concerning RQ1 and the extent that research using or about MMR is occurring in LIS, the main results are reported in Table 4. We identified 373 (89.5%) articles that primarily used mixed methods as part of the research process, and another 44 (10.5%) articles were that were primarily about MMR and related methodological discussions. Addressing RQ2, nearly one-third (n=118, 31.6%) of the 373 articles using mixed methods processes were situated in a health context. Similarly, more than one-third (n=18, 40.9%) of the 44 articles about mixed methods or research methodologies occurred in health contexts. When combined, these health-focused articles comprised 136 published articles (32.6% of the total) related to health or medical sciences within the overarching LIS literature.

Table 4

Number and Percentage of Each Article Type

Type		No. of articles	% of total
All "using" articles		373	89.45%
	Articles using MMR	255	61.15%
	Articles using MMR in health contexts	118	28.30%
All "about" articles		44	10.55%
	Articles about MMR	26	6.24%
	Articles about MMR in health contexts	18	4.32%

MMR over time

We tracked the number of articles published per year to look for developmental trends over the decade (Figure 2). This distribution demonstrates an increasing trend in the use and discussion of mixed methods processes within LIS research. We also combined the articles using MMR with those using MMR in health contexts, to compare them against all of the articles about MMR combined with those about MMR in health contexts. This comparison, shown in figure 3, reveals that the trend in research about MMR is fairly static, and that it is the studies using MMR that drive the overall increasing trend.

Figure 2

Distribution by publication year for all included articles (n=417).

Figure 3

Distribution by publication year comparing articles using MMR (n=373) and articles about MMR (n=44).

Discussion

MMR trends and patterns

As shown in our search strategies for LISTA and WoS we could isolate the total results for LIS generally before including our MMR-related phrases and keywords. We found that proportionally MMR makes up a tiny fraction of the corpus of LIS research literature. Consider that:

● the 354 results found in LISTA represent 0.36% of the 98,343 total LISTA results when searching with search limiters, but not using keywords; and

● the 282 results found in WoS represent 0.52% of the 51,873 total WoS results when searching with search limiters, but not using keywords.

While this is an imprecise measure, our use of disciplinary and other search limiters (i.e., date range, language, document type, publication type) together help us significantly refine the corpus of available, published LIS literature. These figures provide a compelling case for identifying an overall lack of MMR processes within LIS research.

Despite the small number of MMR contributions overall, the upward trend does show some growth in the use of MMR within LIS (see figure 2). We see this as a promising area for future research. However, in contrast with the growth seen via the increase in the number of articles where MMR was used, we found that much fewer articles discussed mixed methods as a research process, and those that did most often occurred in health contexts. The prevalence of articles about MMR has remained relatively static (see figure 3), an indication that, in addition to fostering momentum around expanded use of MMR, there are likely opportunities for further research contributions aimed at discussing mixed methods processes and related meta-research aspects within LIS. As an example, Venkatesh, Brown, and Bala’s (2013) guidelines for conducting MMR in information systems appears to have met, or created, an appetite for contributions about mixed methods, with their article having received nearly 2,000 citations according to Google Scholar at the time this article was drafted. Our methodological review indicates that further scholarly contributions that intentionally and explicitly connect MMR with LIS would be valuable, and likely necessary.

Focusing again on health contexts (RQ2), our analysis shows that health-related mixed methods research appears to be prominent, with 31.6% of articles using MMR and 40.9% of articles about mixed methods or methodologies being situated in a health context. Within the list of top ten publications containing MMR articles there is a substantial representation of MMR with a health focus. Within the top five journals with MMR publications (see table 3), three of these are health-focused, and collectively, these three journals published 92 (22.1%) of the articles we examined. Since EBP in LIS has known connections to evidence based medicine (EBM) and EBP in health settings, perhaps this is unsurprising. However, these results do underscore that health-related research has a strong presence in the LIS literature that either uses or is about mixed methods.

Researchers and practitioners in LIS who are interested in MMR may look to health-related research to determine practices that could help bolster MMR in other topic areas or contexts. For example, O’Cathain, Murphy, and Nicholl (2008) identified that collaboration is often an important part of mixed methods research, emphasizing that MMR in health settings often involves large interdisciplinary or multidisciplinary teams bringing together people with a variety of expertise. It could be the case that engagement with MMR is occurring in health contexts to a large extent due to the involvement of experts within and beyond LIS on these larger interdisciplinary teams. Further exploration of the role of LIS researchers and other information professionals on such MMR teams could provide insights into effective research practices and other lessons learned that could help extend mixed methods approaches (and MMR in EBP) from these health-focused research projects to the broader LIS research community.

Connections to the research methods literature

To place our findings in the context of wider work on research methods we consulted the LIS literature generally, seeking connections between our methodological review of MMR to overall research trends in LIS via a discipline-focused discussion. The literature reveals that the discipline draws heavily on quantitative research approaches and surveys, though there are some signs that this could be changing. Booth and Brice (2004) found that “LIS research typically utilizes designs of limited applicability, such as the user survey” (p. 91), while Koufogiannakis, Slater, and Crumley’s (2004) content analysis of librarianship research found that descriptive research, mainly using a survey or questionnaire method, was the highest proportion of research published (p. 232). Such points have been an ongoing refrain in the field of library and information science.

A decade later, Turcios, Agarwal, and Watkins (2014) demonstrated that surveys were still the most popular research method used. Similarly, Aytac and Slutsky’s (2014) analysis of LIS research published from 2008-2012 found very few studies (1%) using multiple or combined method approaches. Descriptive research and surveys remained the most popular in LIS, with a majority of the studies employing solely quantitative analysis (69%). They predicted an ongoing growth in practitioner research, but cautioned against over-use of descriptive statistical analysis, instead encouraging practitioners “[to] seek out training in more advanced statistical methods” (p. 152).

These and other authors contend that, although there are a variety of research methodologies employed across LIS, mixed methods approaches have not gained adequate recognition in the field. Gauchi Risso’s (2016) analysis of research methods from 1970-2010 similarly showed the prevalence of descriptive methodologies while stating that “LIS needs new methodological developments, which should combine qualitative and quantitative approaches” (p. 74). Likewise, Ullah and Ameen’s (2018) analysis demonstrated a predominance of quantitative, descriptive, and empirical methodologies in LIS, with survey research still being the most widely used method.

All of this points to the need for LIS researchers to give more consideration to, and increase their awareness of, other research approaches, including mixed methods. Wilson (2013) advocated that those who support evidence based practice in LIS would benefit from expanding their methodological approaches to include mixed methods, that approaching “a research question from multiple methodological perspectives in the same study will add a depth and breadth to the findings, and open up options for data collection and analysis” (p. 277).

Studies exploring MMR approaches in LIS continue to show low uptake of MMR. Fidel (2008) found that only 5% of LIS articles employed mixed methods and that “recognition of MMR by name or as a research method was absent from these articles and from the methodological literature in LIS” (p. 265). This 5% figure was also reported by Venkatesh et al. (2013). Chu’s (2015) analysis found somewhat more variety in research methods used in the field, but the overall results underscored a need to continue expanding and developing research methods and their application to LIS. Chu concluded that “more efforts in the form of education, training, and advocacy are needed to promote the use of multiple methods” (p. 40).

Research methods trends have implications for research of specific topics in LIS today. Matusiak’s (2017) analysis of methodologies in information behaviour research reflected the same themes of overall LIS research practices, finding a majority studies were quantitative and used the common approaches (i.e., surveys). This shows a lack of growth in qualitative and mixed methods, ultimately reinforcing the need to increase awareness in LIS about these research areas. Moreover, in exploring the long-standing over-reliance on surveys and quantitative methodologies for research of technology-acceptance models and information systems (IS) generally, Wu (2012) emphasized that “a mixed methods approach combining both qualitative and quantitative techniques deserves more attention from IS researchers” (p. 175). These trends from the wider research methods literature support our findings and confirm the underexplored opportunities for current LIS researchers and practitioners to consider ways to expand their suite of approaches to adopt MMR (and other methods), increasing and enhancing strategies available for collecting, analysing, and using evidence in research and practice.

This is not to say that quantitative methodologies, descriptive research, and survey methods do not have their place, as we know they can be valuable. Koufogiannakis et al. (2004) noted that LIS is not unique in its tendency toward descriptive research, explaining that these approaches are likely ubiquitous in the field because “they are inexpensive and relatively easy to conduct, can be carried out in a short period of time, and the results are generally easy to analyze” (p. 233). Common research approaches such as surveys likely continue to be popular within and beyond LIS precisely because they offer an appropriate means of addressing particular research questions and problems.

Ultimately, it is important for those conducting any research to consider whether and how a particular methodology and the related method(s) are aligned with, and appropriate for, understanding the phenomenon being explored. We see merit in MMR and join our voices with those arguing for increased adoption of MMR processes for EBP, yet we also recognize that MMR is not always the best or most appropriate choice. We strongly agree with scholars such as Venkatesh et al. (2013) that “the decision to conduct mixed methods research should hinge on the research question, purpose, and context” (emphasis in original, p. 22). Nevertheless, the findings from our methodological review of MMR, as well as the experiences described by researcher-practitioners such as ourselves, together inform our assertion that there are untapped opportunities and potential within LIS to continue to go beyond traditional research approaches and increase the adoption of MMR processes. The field can benefit from engagement with MMR as a way to facilitate creative research and to rigorously combine approaches that can and will foster new forms of inquiry.

Limitations

We have presented a broad methodological review examining mixed methods research within LIS published from 2008 through 2018. We did not set out to employ the methods of a focused systematic review or meta-analysis, nor did we complete detailed quantitative or qualitative analyses of all of the included research artifacts, though future research employing these strategies would certainly be valuable. Instead, our comprehensive “wide lens” approach addresses a gap in the extant literature and enables us to better position our findings alongside other methodological and disciplinary discussions. Though we limited our searches to discipline-specific databases available through our current institutional subscriptions and note that both LISTA and WoS provide significant coverage of LIS research publications, these sources are not exhaustive. We acknowledge that other subscription products (e.g., Library & Information Science Abstracts), indexing services (e.g., Google Scholar), web search engines, and other tools may reveal additional published and grey literature that are relevant. Also, we note that though it appears to be the most common terminology used today, the term mixed methods research is not universally used across the discipline. Our search strategies focused on phrases rather than keywords to reflect the reality that MMR studies are sometimes mislabelled, and that this term may not appear on mixed methods work at all. This leads to the possibility that the growing trend identified in the results could be due to improvements toward consistent labelling strategies and terminology for MMR that are otherwise difficult to capture. Like all research projects, this study may have benefitted from a larger research team, particularly for greater access to search indices and sources, increased scope including grey literature and conference materials, and additional experts participating in verification and consensus steps.

Conclusions

Our findings show that there are still untapped opportunities to extend scholarly contributions about and using mixed methods in research for library and information science contexts, and further confirms findings from the wider LIS research methods literature. It is our hope that by outlining the following recommendations for developing MMR in EBP, we can encourage other researchers and practitioners in their developing their understanding of mixed methods processes, ideally embracing the benefits and opportunities that MMR offers.

Recommendations

The methodological review of the MMR literature, as well as the authors’ own experiences conducting mixed methods research (Hayman, Smith, & Storrs, 2019; Smith, 2016), inform our outline of current needs and related recommendations to extend the development and application of mixed methods in LIS. One recommendation is to encourage researchers and researcher-practitioners to undertake MMR when appropriate. We echo the calls from Chu (2015) and others to promote further understanding of MMR through education, training, and advocacy. Efforts to expand engagement with MMR through informal, non-formal, and formal education, including in graduate curriculum for library and information schools, could help to develop scholarship not just using MMR, but also about mixed methods processes and aspects of meta-research. Intentionally integrating such pedagogical strategies aligns with Crumley and Koufogiannakis’ (2002) assertion that learning research skills is “essential for the growth of EBL [evidence based librarianship] within the entire profession” (p. 69). We note that this need to support the teaching and learning of research methods that includes MMR is not limited to LIS. Tashakkori and Teddlie (2010) created their handbook on MMR for the social sciences broadly, and the field of education specifically, based on their practical experience working with graduate students on research methods training. They include a section with specific recommendations for pedagogy since this topic emerged “as one of the most difficult and controversial areas in mixed methods” (p. xi). Given these complexities are widespread, LIS could certainly take advantage of emerging and established educational developments through cross-disciplinary collaborations with other areas such as health and education. Our findings showing the prevalence of MMR research in health-related contexts makes this point clear.

Resources such as handbooks and toolkits from mixed methodologists can be helpful. A related recommendation is for graduate-level research methods courses to explore ways to enhance their coverage of mixed methods research. While some graduate programs may increasingly recognize the use of MMR in the research process, further scaffolding and building of expertise within and across disciplines, including those in LIS, is warranted as a means of mitigating the challenges of MMR with the goal of realizing the benefits. As MMR evolves, the creation of courses and open resources that outline the theoretical, empirical, and practical considerations for mixed methods and its designs that can be easily accessed beyond the academy would also be beneficial in this regard. So would continuous professional development (CPD) opportunities on evolving research methods and MMR – for example, CPD connected to professional associations, conferences, and journals – that provide venues for LIS researchers and practitioners at all levels who wish to reflect the principles of EBP and expand their methodological repertoire.

Summary

In returning to Fidel’s question of whether LIS is “there yet” in engaging with MMR, we find that while there has been some growth in the use of mixed methods over the past decade, our methodological review demonstrates that MMR still represents only a small fraction of current LIS literature. These findings indicate that further contributions about MMR processes and approaches are still needed, including those explicitly connecting research with practice. Our results also show some momentum in MMR use, with an observable increase in the number of publications using mixed methods in LIS during the decade in question, and that there is room for future research to explore this trend. Health research contexts have a particularly strong presence in scholarship using and about MMR in LIS, highlighting an opportunity to translate lessons learned about MMR and practice from health-focused areas into other LIS settings. Based on our findings, we recommend that the LIS research community look to actively facilitate greater engagement with mixed methods, so that wider awareness and understanding of MMR can be fostered through educational development initiatives that build pedagogical strategies and resources for MMR, especially those supporting graduate programs and bridging academic and practitioner communities. Enhancing ways to develop and apply mixed methods research in LIS contexts in ways that take advantage of the affordances of MMR will benefit evidence based library and information practice.

Data availability statement

A dataset (Hayman & Smith, 2019) including the combined 636 citations exported from LISTA and WoS is available in BibTex (.bib) format.

References

Aytac, S., & Slutsky, B. (2014). Published librarian research, 2008 through 2012: Analyses and perspectives. Collaborative Librarianship, 6(4), 147-159. Retrieved from https://digitalcommons.du.edu/collaborativelibrarianship/vol6/iss4/6/

Booth, A., & Brice, A. (Eds.). (2004). Evidence-based practice for information professionals: A handbook. London, UK: Facet.

Chu, H. (2015). Research methods in library and information science: A content analysis. Library & Information Science Research, 37(1), 36-41. https://doi.org/10.1016/j.lisr.2014.09.003

Connaway, L. S., & Radford, M. L. (2017). Research methods in library and information science. Santa Barbara, CA: Libraries Unlimited.

Creswell, J. W. (2008). Mixed methods research. In L. M. Given (Ed.), The SAGE encyclopedia of qualitative research methods (pp. 527-530). https://doi.org/10.4135/9781412963909.n269

Creswell, J. W., & Hirose, M. (2019). Mixed methods and survey research in family medicine and community health. Family Medicine and Community Health, 7(2), e000086.

https://doi.org/10.1136/fmch-2018-000086

Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). Los Angeles, CA: SAGE.

Crumley, E., & Koufogiannakis, D. (2002). Developing evidence-based librarianship: Practical steps for implementation. Health Information and Libraries Journal, 19(2), 61-70. https://doi.org/10.1046/j.1471-1842.2002.00372.x

Elsevier. (n.d.). A guide for writing scholarly articles or reviews for the Educational Research Review. Retrieved from https://www.elsevier.com/__data/promis_misc/edurevReviewPaperWriting.pdf

Fidel, R. (2008). Are we there yet? Mixed methods research in library and information science. Library & Information Science Research, 30(4), 265-272. https://doi.org/10.1016/j.lisr.2008.04.001

Gauchi Risso, V. (2016). Research methods used in library and information science during the 1970-2010. New Library World, 117(1/2), 74-93. https://doi.org/10.1108/NLW-08-2015-0055

Gorman, G. E., & Clayton, P. (2005). Qualitative research for the information professional: A practical handbook (2nd ed.). London, UK: Facet.

Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26(2), 91-108.

https://doi.org/10.1111/j.1471-1842.2009.00848.x

Hayman, R., & Smith, E. E. (2019). Mixed methods in library and information science bibliographic records (2008-2018) [Data set]. Edmonton, Canada: UAL Dataverse, ver. 1.0, https://doi.org/10.7939/dvn/cxuw6a

Hayman, R., Smith, E. E., & Storrs, H. (2019). Information behaviour of undergraduate students using Facebook Confessions for educational purposes. Information Research, 24(3), paper rails1809.

http://informationr.net/ir/24-3/rails/rails1809.html

Huynh, T., Hatton-Bowers, H., & Smith, M. H. (2019). A critical methodological review of mixed methods designs used in mindfulness research. Mindfulness, 10(5), 786-798.

https://doi.org/10.1007/s12671-018-1038-5

Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112-133. https://doi.org/10.1177/1558689806298224

Koufogiannakis, D., Slater, L., & Crumley, E. (2004). A content analysis of librarianship research. Journal of Information Science, 30(3), 227-239. https://doi.org/10.1177/0165551504044668

Matusiak, K. K. (2017). Studying information behavior of image users: An overview of research methodology in LIS literature, 2004-2015. Library & Information Science Research, 1(39), 53-60. https://doi.org/10.1016/j.lisr.2017.01.008

O’Cathain, A., Murphy, E., & Nicholl, J. (2008). Multidisciplinary, interdisciplinary, or dysfunctional? Team working in mixed-methods research. Qualitative Health Research, 18(11), 1574-1585. https://doi.org/10.1177/1049732308325535

Onwuegbuzie, A., Leech, N. & Collins, K. (2011). Innovative qualitative data collection techniques for conducting literature reviews/research syntheses. In M. Williams & W. P. Vogt (Eds.), The SAGE handbook of innovation in social research methods (pp. 182-204). London, UK: SAGE. https://doi.org/10.4135/9781446268261.n13

Plano Clark, V., & Ivankova, N. (2016). Mixed methods research: A guide to the field. Thousand Oaks, CA: SAGE. https://doi.org/10.4135/9781483398341

Smith, E. E. (2016). “A real double-edged sword:” Undergraduate perceptions of social media in their learning. Computers & Education, 103, 44-58. https://doi.org/10.1016/j.compedu.2016.09.009

Tashakkori, A., & Teddlie, C. (2010). The SAGE handbook of mixed methods in social & behavioural research (2nd ed.). Los Angeles, CA: SAGE. https://doi.org/10.4135/9781506335193

Turcios, M. E., Agarwal, N. K., & Watkins, L. (2014). How much of library and information science literature qualifies as research? The Journal of Academic Librarianship, 40(5), 473-479. https://doi.org/10.1016/j.acalib.2014.06.003

Ullah, A., & Ameen, K. (2018). Account of methodologies and methods applied in LIS research: A systematic review. Library & Information Science Research, 40(1), 53-60. https://doi.org/10.1016/j.lisr.2018.03.002

Venkatesh, V., Brown, S. A., & Bala, H. (2013). Bridging the qualitative-quantitative divide: Guidelines for conducting mixed methods research in information systems. MIS Quarterly, 37(1), 21-54. https://doi.org/10.25300/MISQ/2013/37.1.02

Wilson, V. (2013). Research methods: Mixed methods research. Evidence Based Library and Information Practice, 8(2), 275-277. https://doi.org/10.18438/B8801M

Wilson, V. (2014). Research methods: Triangulation. Evidence Based Library and Information Practice, 9(1), 74-75. https://doi.org/10.18438/B8WW3X

Wu, P. F. (2012). A mixed methods approach to technology acceptance research. Journal of the Association for Information Systems, 13(3), 172-187. https://doi.org/10.17705/1jais.00287

Appendix A

List of publication titles and corresponding number of articles published that were included in the findings (n=417)

Publication title	No. of articles
Journal of Medical Internet Research	62
Qualitative Health Research	20
Information Research	10
Journal of the American Medical Informatics Association	10
Journal of the Association for Information Science & Technology (formerly Journal of the American Society for Information Science & Technology)	10
Aslib Journal of Information Management (formerly Aslib Proceedings)	8
Internet & Higher Education	8
Journal of Health Communication	8
Information Technology & People	7
Journal of Librarianship & Information Science	7
MIS Quarterly	7
South African Journal of Information Management	7
College & Research Libraries	6
Education for Information	6
Evidence Based Library & Information Practice	6
First Monday	6
Information Development	6
Journal of Documentation	6
Qualitative & Quantitative Methods in Libraries	6
Electronic Library	5
Information, Communication & Society	5
Journal of the Association for Information Systems	5
Mousaion	5
New Review of Academic Librarianship	5
Technology, Pedagogy & Education	5
Health Informatics Journal	4
Health Information & Libraries Journal	4
Information and Learning Science (formerly New Library World)	4
International Journal of Information Management	4
Journal of Enterprise Information Management	4
Journal of the Medical Library Association	4
Library & Information Science Research	4
Library Management	4
Public Library Quarterly	4
South African Journal of Libraries & Information Science	4
African Journal of Library, Archives & Information Science	3
Canadian Journal of Information & Library Sciences	3
European Journal of Information Systems	3
IFLA Journal	3
Information & Management	3
International Information & Library Review	3
Internet Research	3
Journal of Education for Library & Information Science	3
Journal of Information & Knowledge Management	3
Journal of Nonprofit & Public Sector Marketing	3
Journal of Strategic Information Systems	3
Library Hi Tech	3
Library Trends	3
Online Information Review	3
portal: Libraries & The Academy	3
Reference Services Review	3
Research Evaluation	3
Telematics and Informatics	3
Transforming Government: People Process and Policy	3
Information Processing & Management	2
Information Systems Journal	2
Information Technology for Development	2
Innovation	2
International Journal of Information & Communication Technology Education	2
Journal of Academic Librarianship	2
Journal of the Australian Library & Information Association (formerly Australian Library Journal)	2
Journal of Information Science	2
Journal of Information Technology & Politics	2
Journal of Organizational & End User Computing	2
Journal of Technology in Human Services	2
Library Review	2
Libri: International Journal of Libraries & Information Services	2
New Zealand Library & Information Management Journal	2
Open Learning	2
School Libraries Worldwide	2
Social Science Computer Review	2
Accountability in Research: Policies & Quality Assurance	1
Archival Science	1
Archives & Manuscripts	1
Behaviour & Information Technology	1
College & Undergraduate Libraries	1
Communications in Information Literacy	1
Community & Junior College Libraries	1
Computers in the Schools	1
Data Base for Advances in Information Systems	1
Data Technologies and Applications	1
Electronic Journal of Knowledge Management	1
Hypothesis: Journal of the Research Section of MLA	1
IASSIST Quarterly	1
Informatics for Health & Social Care	1
Information & Organization	1
Information Discovery and Delivery	1
Information Polity: The International Journal of Government & Democracy in the Information Age	1
Information Services & Use	1
Information Society	1
Information Systems Research	1
Information Technology & Management	1
International Journal of Computer-supported Collaborative Learning	1
International Journal of Electronic Government Research	1
International Journal of Technology and Human Interaction	1
International Journal of Web Based Communities	1
Journal of Access Services	1
Journal of Information Systems Education	1
Journal of Information Technology	1
Journal of Informetrics	1
Journal of Intellectual Capital	1
Journal of Knowledge Management	1
Journal of Library & Information Services in Distance Learning	1
Journal of Library Administration	1
Journal of Library Metadata	1
Journal of Scholarly Publishing	1
Knowledge Organization	1
Learned Publishing	1
Library Philosophy & Practice	1
Music Reference Services Quarterly	1
New Review of Information Networking	1
Pakistan Library & Information Science Journal	1
Partnership: The Canadian Journal of Library & Information Practice & Research	1
Performance Measurement & Metrics	1
Public Services Quarterly	1
Publications	1
Publishing Research Quarterly	1
Reference & User Services Quarterly	1
Teacher Librarian	1
Universal Access in the Information Society	1
Vine: The Journal of Information & Knowledge Management Systems	1
Total no. of articles	417