key: cord-269015-850bvb9t authors: Östlund, Ulrika; Kidd, Lisa; Wengström, Yvonne; Rowa-Dewar, Neneh title: Combining qualitative and quantitative research within mixed method research designs: A methodological review date: 2011-03-31 journal: International Journal of Nursing Studies DOI: 10.1016/j.ijnurstu.2010.10.005 sha: doc_id: 269015 cord_uid: 850bvb9t Abstract Objectives It has been argued that mixed methods research can be useful in nursing and health science because of the complexity of the phenomena studied. However, the integration of qualitative and quantitative approaches continues to be one of much debate and there is a need for a rigorous framework for designing and interpreting mixed methods research. This paper explores the analytical approaches (i.e. parallel, concurrent or sequential) used in mixed methods studies within healthcare and exemplifies the use of triangulation as a methodological metaphor for drawing inferences from qualitative and quantitative findings originating from such analyses. Design This review of the literature used systematic principles in searching CINAHL, Medline and PsycINFO for healthcare research studies which employed a mixed methods approach and were published in the English language between January 1999 and September 2009. Results In total, 168 studies were included in the results. Most studies originated in the United States of America (USA), the United Kingdom (UK) and Canada. The analytic approach most widely used was parallel data analysis. A number of studies used sequential data analysis; far fewer studies employed concurrent data analysis. Very few of these studies clearly articulated the purpose for using a mixed methods design. The use of the methodological metaphor of triangulation on convergent, complementary, and divergent results from mixed methods studies is exemplified and an example of developing theory from such data is provided. Conclusion A trend for conducting parallel data analysis on quantitative and qualitative data in mixed methods healthcare research has been identified in the studies included in this review. Using triangulation as a methodological metaphor can facilitate the integration of qualitative and quantitative findings, help researchers to clarify their theoretical propositions and the basis of their results. This can offer a better understanding of the links between theory and empirical findings, challenge theoretical assumptions and develop new theory. Objectives: It has been argued that mixed methods research can be useful in nursing and health science because of the complexity of the phenomena studied. However, the integration of qualitative and quantitative approaches continues to be one of much debate and there is a need for a rigorous framework for designing and interpreting mixed methods research. This paper explores the analytical approaches (i.e. parallel, concurrent or sequential) used in mixed methods studies within healthcare and exemplifies the use of triangulation as a methodological metaphor for drawing inferences from qualitative and quantitative findings originating from such analyses. Design: This review of the literature used systematic principles in searching CINAHL, Medline and PsycINFO for healthcare research studies which employed a mixed methods approach and were published in the English language between January 1999 and September 2009. Results: In total, 168 studies were included in the results. Most studies originated in the United States of America (USA), the United Kingdom (UK) and Canada. The analytic approach most widely used was parallel data analysis. A number of studies used sequential data analysis; far fewer studies employed concurrent data analysis. Very few of these studies clearly articulated the purpose for using a mixed methods design. The use of the methodological metaphor of triangulation on convergent, complementary, and divergent results from mixed methods studies is exemplified and an example of developing theory from such data is provided. Conclusion: A trend for conducting parallel data analysis on quantitative and qualitative data in mixed methods healthcare research has been identified in the studies included in this review. Using triangulation as a methodological metaphor can facilitate the integration of qualitative and quantitative findings, help researchers to clarify their theoretical propositions and the basis of their results. This can offer a better understanding of the links between theory and empirical findings, challenge theoretical assumptions and develop new theory. ß 2010 Elsevier Ltd. All rights reserved. There is a lack of pragmatic guidance in the research literature as how to combine qualitative and quantitative approaches and how to integrate qualitative and quantitative findings. Analytical approaches used in mixed-methods studies differ on the basis of the sequence in which the components occur and the emphasis given to each, e.g. parallel, sequential or concurrent. A trend for conducting parallel analysis on quantitative and qualitative data in healthcare research is apparent within the literature. Using triangulation as a methodological metaphor can facilitate the integration of qualitative and quantitative findings and help researchers to clearly present both their theoretical propositions and the basis of their results. Using triangulation as a methodological metaphor may also support a better understanding of the links between theory and empirical findings, challenge theoretical assumptions and aid the development of new theory. Mixed methods research has been widely used within healthcare research for a variety of reasons. The integration of qualitative and quantitative approaches is an interesting issue and continues to be one of much debate (Bryman, 2004; Morgan, 2007; Onwuegbuzie and Leech, 2005) . In particular, the different epistemological and ontological assumptions and paradigms associated with qualitative and quantitative research have had a major influence on discussions on whether the integration of the two is feasible, let alone desirable (Morgan, 2007; Sale et al., 2002) . Proponents of mixed methods research suggest that the purist view, that quantitative and qualitative approaches cannot be merged, poses a threat to the advancement of science (Onwuegbuzie and Leech, 2005) and that while epistemological and ontological commitments may be associated with certain research methods, the connections are not necessary deterministic (Bryman, 2004) . Mixed methods research can be viewed as an approach which draws upon the strengths and perspectives of each method, recognising the existence and importance of the physical, natural world as well as the importance of reality and influence of human experience (Johnson and Onquegbuzie, 2004) . Rather than continue these debates in this paper, we aim to explore the approaches used to integrate qualitative and quantitative data within healthcare research to date. Accordingly, this paper focuses on the practical issues of conducting mixed methods studies and the need to develop a rigorous framework for designing and interpreting mixed methods studies to advance the field. In this paper, we will attempt to offer some guidance for those interested in mixed methods research on ways to combine qualitative and quantitative methods. The concept of mixing methods was first introduced by Jick (1979) , as a means for seeking convergence across qualitative and quantitative methods within social science research (Creswell, 2003) . It has been argued that mixed methods research can be particularly useful in healthcare research as only a broader range of perspectives can do justice to the complexity of the phenomena studied (Clarke and Yaros, 1988; Foss and Ellefsen, 2002; Steckler et al., 1992) . By combining qualitative and quantitative findings, an overall or negotiated account of the findings can be forged, not possible by using a singular approach (Bryman, 2007) . Mixed methods can also help to highlight the similarities and differences between particular aspects of a phenomenon (Bernardi et al., 2007) . Interest in, and expansion of, the use of mixed methods designs have most recently been fuelled by pragmatic issues: the increasing demand for cost effective research and the move away from theoretically driven research to research which meets policymakers' and practitioners' needs and the growing competition for research funding (Brannen, 2009; O'Cathain et al., 2007) . Tashakkori and Creswell (2007) broadly define mixed methods research as ''research in which the investigator collects and analyses data, integrates the findings and draws inferences using both qualitative and quantitative approaches' ' (2007:3) . In any mixed methods study, the purpose of mixing qualitative and quantitative methods should be clear in order to determine how the analytic techniques relate to one another and how, if at all, the findings should be integrated (O'Cathain et al., 2008; Onwuegbuzie and Teddlie, 2003) . It has been argued that a characteristic of truly mixed methods studies are those which involve integration of the qualitative and quantitative findings at some stage of the research process, be that during data collection, analysis or at the interpretative stage of the research (Kroll and Neri, 2009 ). An example of this is found in mixed methods studies which use a concurrent data analysis approach, in which each data set is integrated during the analytic stage to provide a complete picture developed from both data sets after data has been qualitised or quantitised (i.e. where both forms of data have been converted into either qualitative or quantitative data so that it can be easily merged) (Onwuegbuzie and Teddlie, 2003) . Other analytic approaches have been identified including; parallel data analysis, in which collection and analysis of both data sets is carried out separately and the findings are not compared or consolidated until the interpretation stage, and finally sequential data analysis, in which data are analysed in a particular sequence with the purpose of informing, rather than being integrated with, the use of, or findings from, the other method (Onwuegbuzie and Teddlie, 2003) . An example of sequential data analysis might be where quantitative findings are intended to lead to theoretical sampling in an in depth qualitative investigation or where qualitative data is used to generate items for the development of quantitative measures. When qualitative and quantitative methods are mixed in a single study, one method is usually given priority over the other. In such cases, the aim of the study, the rationale for employing mixed methods, and the weighting of each method determine whether, and how, the empirical findings will be integrated. This is less challenging in sequential mixed methods studies where one approach clearly informs the other, however, guidance on combining qualitative and quantitative data of equal weight, for example, in concurrent mixed methods studies, is rather less clear (Foss and Ellefsen, 2002) . This is made all the more challenging by a common flaw which is to insufficiently and inexplicitly identify the relationships between the epistemological and methodological concepts in a particular study and the theoretical propositions about the nature of the phenomena under investigation (Kelle, 2001) . One approach to combining different data of equal weight and which facilitate clear identification of the links between the different levels of theory, epistemology, and methodology could be to frame triangulation as a 'methodological metaphor', as argued by Erzberger and Kelle (2003) . This can help to; describe the logical relations between the qualitative and quantitative findings and the theoretical concepts in a study; demonstrate the way in which both qualitative and quantitative data can be combined to facilitate an improved understanding of particular phenomena; and, can also be used to help generate new theory (Erzberger and Kelle, 2003 ) (see Fig. 1 ). The points of the triangle represent theoretical propositions and empirical findings from qualitative and quantitative data while the sides of the triangle represent the logical relationships between these propositions and findings. The nature and use of the triangle depends upon the outcome from the analysis, whether that be convergent, where qualitative and quantitative findings lead to the same conclusion; complementary, where qualitative and quantitative results can be used to supplement each other or; divergent, where the combination of qualitative and quantitative results provides different (and at times contradictory) findings. Each of these outcomes requires a different way of using the triangulation metaphor to link theoretical propositions to empirical findings (Erzberger and Kelle, 2003) . In the following paper, we identify the analytical approaches used in mixed methods healthcare research and exemplify the use of triangulation (Erzberger and Kelle, 2003) as a methodological metaphor for drawing inferences from qualitative and quantitative findings. Papers reporting on mixed methods studies within healthcare research were reviewed to (i) determine the type of analysis approach used, i.e. parallel, concurrent, or sequential data analysis and, (ii) identify studies which could be used to illustrate the use of the methodological metaphor of triangulation suggested by Erzberger and Kelle (2003) . Four papers were selected to illustrate the application of the triangulation metaphor on complementary, convergent and divergent outcomes and to develop theory. This literature review has used systematic principles (Cochrane, 2009 , Khan, 2001 to search for mixed methods studies within healthcare research. The first search was conducted in September 2009 in the data bases CINAHL, Medline and PsycINFO on papers published in English language between 1999 and 2009. To identify mixed methods studies, the search terms (used as keywords and where possible as MeSH terms) were: ''mixed methods'', ''mixed research methods'', ''mixed research'', ''triangulation'', ''complementary methods'', ''concurrent mixed analysis'' and ''multi-strategy research.'' These terms were searched individually and then combined (with OR). This resulted in 1896 hits in CINAHL, 1177 in Medline and 1943 in PsycINFO. To focus on studies within, or relevant to, a healthcare context the following search terms were used (as keywords or as MeSH terms and combined with OR): ''health care research''; ''health services research''; and ''health''. These limits applied to the initial search (terms combined with AND) resulted in 205 hits in Medline and 100 hits in PsycINFO. Since this combination in CINAHL only limited the search results to 1017; a similar search was conducted but without using the search term triangulation to capture mixed methods papers; resulting in 237 hits. In CINAHL the search result on 1017 papers was further limited by (Erzberger and Kelle, 2003) using ''interventions'' as a keyword resulting in 160 papers also selected to be reviewed. Moreover; in Medline the mixed methods data set was limited by the MeSH term ''research'' resulting in 218 hits and in PsycINFO with ''intervention'' as keyword or MeSH term resulting in 178 hits. When duplicates were removed the total numbers of papers identified were 843. The abstracts were then reviewed by each author and those identified as relevant to the review were selected to be retrieved and reviewed in full text. Papers were selected based on the following inclusion criteria: empirical studies; published in peer review journals; healthcare research (for the purpose of this paper defined as any study focussing on participants in receipt, or involved in the delivery, of healthcare or a study conducted within a healthcare setting, e.g. different kinds of care, health economics, decision making, and professionals' role development); and using mixed methods (defined as a study in which both qualitative and quantitative data were collected and analysed (Halcomb et al., 2009b) . To maintain rigour, a random sample (10%) of the full text papers was reviewed jointly by two authors. Any disagreements or uncertainties that arose between the reviewers regarding their inclusion or in determining the type of analytic approach used were resolved through discussion between the authors. In addition to the criteria outlined above, papers were excluded if the qualitative element constituted a few openended questions in a questionnaire, as we would agree with previous authors who have argued such studies do not strictly constitute a mixed methods design (Kroll and Neri, 2009) . Papers were also excluded if they could not be retrieved in full text via the library services at the University of Edinburgh, Glasgow Caledonian University or the Karolinska Institutet, or did not adequately or clearly describe their analytic strategy, for example, failing to report how the qualitative and quantitative data sets were analysed individually and, where relevant, how these were integrated. See Table 1 for reasons for the exclusion of subsequent papers. A second search was conducted within the databases of Medline, PsychInfo and Cinahl to identify studies which have specifically used Erzberger and Kelle's (2003) triangulation metaphor to frame the description and interpretation of their findings. The term 'triangulation metaphor' (as keywords) and author searches on 'Christian Erzberger' and 'Udo Kelle' were conducted. Three papers, published by Christian Erzberger and Udo Kelle, were identified in the PsychInfo databases but none of these were relevant to the purpose of this review. There were no other relevant papers identified in the other two databases. 168 Papers were included in the final review and reviewed to determine the type of mixed analysis approach used, i.e. parallel, concurrent, or sequential mixed analysis. Four of these papers (identified from the first search on mixed methods studies and healthcare research) were also used to exemplify the use of the methodological metaphor of triangulation (Erzberger and Kelle, 2003) . Data was extracted from included papers accordingly in relation to these purposes. In total, 168 papers were included in our review. The majority of these studies originated in the USA (n = 63), the UK (n = 39) and Canada (n = 19), perhaps reflecting the considerable interest and expertise in mixed methods research within these countries. The focus of the studies included in the review varied significantly and the populations studied included both patients and healthcare professionals. Table 2 illustrates the types of analytic approaches adopted in each of the studies included in the review. The most widely used analytic approach (n = 98) was parallel analysis (Creswell and Plano Clark, 2007) . However, very few of the studies employing parallel analysis clearly articulate their purpose for mixing qualitative and quantitative data, the weighting (or priority) given to the qualitative and quantitative data or the expected outcomes from doing so, mirroring previous research findings (O'Cathain et al., 2008) . The weighting, or priority, of the qualitative and quantitative data in a mixed methods study is dependent upon various factors including; the aims of the study and whether the purpose is, for example, to contextualise quantitative data using qualitative data or to use qualitative data to inform a larger quantitative approach such as a survey. Nonetheless, the omission of this statement makes it difficult to determine which data set the conclusions have been drawn from and the role of, or emphasis on, each approach. Therefore, is of importance for authors to clearly state this in their papers (Creswell and Plano Clark, 2007) . A number of studies had also used sequential data analysis (n = 46), where qualitative approaches were visibly used to inform the development of both clinical tools (e.g. and research measures and surveys (e.g. or where quantitative surveys were supplemented by and issues further explored using qualitative approaches (e.g. . Table 1 Reasons for exclusion. Number of papers Not empirical study/not published in peer reviewed journal/full text unavailable 296 Not mixed method report/both data sets not presented or retrievable 287 Not healthcare research 88 Analysis/findings not clearly described a 4 a Due to lack of clarity in these papers method and/or result presentations it was not possible to determine what analysis technique had been used. Included papers illustrating their analytical approach and country of origin. Papers used a sequential analytic approach (n = 46) (3) Most notably, with only 20 included studies using a concurrent approach to data analysis, this was the least common design employed. Compared to the studies using a parallel or sequential approach, the authors of concurrent studies more commonly provided an explanation for their purpose of using a mixed methods design in their study, e.g. how it addressed a gap or would facilitate and advance the state of knowledge (e.g. . Despite this, there remained a lack of clarity within these studies about the weighting given to, and priority of, each method. Consequently, the importance and relevance of the findings produced by each approach and how these have informed their conclusions and interpretation is lacking. In four of the included papers a combination of approaches to data analysis (i.e. sequential and concurrent, parallel and concurrent, or sequential and parallel) were used. In the next section, we have selected papers to illustrate the methodological metaphor of triangulation (Erzberger and Kelle, 2003) . We have selected four papers from our review to illustrate how the methodological metaphor of triangulation (Erzberger and Kelle, 2003) can be applied to mixed methods studies. Each of these studies has been used to illustrate how the metaphor of triangulation can be applied to studies producing: (i) complementary findings, (ii) convergent findings, and (iii) divergent findings. In the following section, we demonstrate how the application of the metaphor can be used as a framework both to develop theory and to facilitate the interpretation of the findings from mixed methods studies and their conclusions in each of these scenarios, and how using the metaphor can help to promote greater clarity of the study's purpose, its theoretical propositions, and the links between data sets. To exemplify the use of the methodological metaphor of triangulation (Erzberger and Kelle, 2003) for drawing inferences from complementary results, we have drawn on the results of a UK based study by (Fig. 2) . In the UK, members of district nursing teams (DNs) provide most nursing care to people at home in the last year of life. Following concerns that inadequate education might limit the confidence of some DNs to support patients and their carers' at home, and that low home death rates may in part be related to this, the Department of Health (DH) identified good examples of palliative care educational initiatives for DNs and invested in a 3-year national education and support programme in the principles and practice of palliative care. Shipman et al.'s study evaluates whether the programme had measurable effects on DN knowledge and confidence in competency in the principles and practice of palliative care. The study had two parts, a summative (concerned with outcomes) quantitative component which included 'before and after' postal questionnaires which measured effects on DNs' (n = 1280) knowledge, confidence and perceived competence in the principles and practice of palliative care and a formative (concerned with process) qualitative component which included semi-structured focus groups and interviews with a sub-sample of DNs (n = 39). While their theoretical proposition may not be explicitly stated by the authors, there is clearly an implicit theoretical proposition that the educational intervention would improve DNs knowledge and confidence (theoretical proposition 1, Fig. 2 ). This was supported by the quantitative findings which showed significant improvement in the district nurses confidence in their professional competence post intervention. Qualitative results supported and complemented the quantitative findings as the district nurses described several benefits from the program Improved confidence in competence emanate from increased understanding of factors contributing to complex problems and underlying reasons for symptoms. Quantitative data showed improvement in the DNs confidence in competence post intervention. including greater confidence in tackling complex problems and better communication with patient and carers' because of greater understanding of the reasons for symptoms. Thus, a complementary theoretical proposition (theoretical proposition 2, Fig. 2 ) can be deduced from the qualitative findings: the DN's better understanding of factors contributing to complex problems and underlying reasons for symptoms led to improved confidence in competence raised from district nurses increased understanding. Fig. 2 illustrates the theoretical propositions, the empirical findings from qualitative and quantitative data and the logical relationships between these. Theoretical proposition 1 is supported by the quantitative findings. From qualitative findings, a complementary theoretical proposition (theoretical proposition 2) can be stated explaining the process that led to the DNs improved confidence in competence. To illustrate how the methodological metaphor of triangulation can be used to draw inferences from convergent findings, we have drawn on the example of a Danish study by (Fig. 3) . This study was conducted to explore experiences of group cohesion and changes in quality of life (QoL) among people (n = 55) who participated in a weekly physical exercise intervention (for six weeks) during treatment for cancer. The study, conducted in a Danish hospital, involved the use of structured QoL questionnaires, administered at baseline and post intervention (at six weeks) to determine changes in QoL and health status, and qualitative focus groups, conducted post intervention (at six weeks), to explore aspects of cohesion within the group. With regards to the theoretical proposition of the study (Fig. 3) , group cohesion was seen as essential to understand the processes within the group that facilitated the achievement of desired outcomes and the satisfaction of affective needs as well as promoting a sense of belonging to the group itself. This proposition was deductively tested in an intervention where patients exercised in mixed gender groups of seven to nine members during a nine hour weekly session over a six week period and was supported by both the empirical quantitative and qualitative findings. The quantitative data showed significant improvements in peoples' emotional functioning, social functioning and mental health. The qualitative data showed how the group setting motivated the individuals to pursue personal endeavors beyond physical limitations, that patients used each others as role models during 'down periods' and how exercising in a group made individuals feel a sense of obligation to train and to do their best. This subsequently helped to improve their social functioning which in turn satisfied their affective needs, improving their improved emotional functioning and mental health. Fig. 3 illustrates the theoretical propositions, empirical findings from qualitative and quantitative data and the logical relationships between these. Both the quantitative and qualitative findings, demonstrating improvements in participants' emotional and social functioning and their mental health, can be attributed to the nature of group cohesion within the programme as expected. Qualitative and quantitative results that seem to contradict each other are often explained as resulting from methodological error. However, instead of a methodological flaw, a divergent result could be a consequence of the inadequacy of the theoretical concepts used. This may indicate the need for changing or developing the theoretical concepts involved (Erzberger and Kelle, 2003) . The following example of using the methodological metaphor of triangulation (Erzberger and Kelle, 2003) for drawing inferences from divergent results is intended as an example rather than an attempt to change the theoretical concept involved. In a study by (Fig. 4) , some results were found to be divergent which was explained as resulting from the use of Theoretical level Quantitative data showed significant improvement in relation to emotional functioning, social functioning and mental health. Qualitative data showed the group setting motivated individuals to pursue personal endeavors beyond physical limitations and made individuals feel a sense of obligation to train and to do their best, thereby improving their social and emotional functioning and mental health. Group cohesion is essential in facilitating the achievement of desired outcomes (instrumental objectives) and the satisfaction of affective needs as well as a sense of belonging. Fig. 3 . Illustrating the use of triangulation (Erzberger and Kelle, 2003) on convergent results in the study by . inadequate questionnaires. We do not wish to critique their conclusion; rather we intend to simply offer an alternative interpretation for their findings. The study aimed to explore family carers' expectations and experiences of respite services provided by one independent hospice in North England. This hospice provides inpatient respite beds specifically for planned respite admission for a two-week period. Referrals were predominated from general practitioners and patients and their carers were offered respite care twice a year, during the study this was reduced to once a year for each patient. Data was collected prior to respite admission and post respite care by semi-structured interviews and using the Relative Stress Scale inventory (RSSI), a validated scale to measure relative distress in relation to caring. Twenty-five carers were included but pre-and post-data were completed by 12 carers. Qualitative data was analysed by using a process of constant comparison and quantitative data by descriptive and comparative statistical analysis. No clear theoretical proposition was stated by the authors, but from the definition of respite care it is possible to deduce that 'respite care is expected to provide relief from care-giving to the primary care provider' (theoretical proposition 1, Fig. 4 ). This proposition was tested quantitatively by pre-and post-test using the RSSI showing that the majority of carers experienced either a negative or no change in scores following the respite stay (no test of significance was stated). Accordingly, the theoretical proposition was not supported by the quantitative empirical data. The qualitative empirical results, however, were supportive in showing that most of the carers considered respite care to be important as it enabled them to have a break and a rest from ongoing care-responsibilities. From this divergent empirical data it could be suggested changing or developing the original theoretical proposition. It seems that respite care gave the carers relief from their care-responsibilities but not from the distress carers experienced in relation to caring (measured by the used scale). We would therefore suggest that in order to lessen distress related to caring, other types of support is also needed which would change the theoretical proposition as suggested (theoretical proposition 2). Fig. 4 illustrates the theoretical propositions, empirical findings from qualitative and quantitative data and the logical relationships between these. Theoretical proposition 1 was not supported by the quantitative findings (indicated in Fig. 4 by the broken arrow), but the qualitative findings supported this proposition. From these divergent empirical findings, the theoretical proposition could accordingly be changed and developed. Respite care seemed to provide relief from carers' on-going careresponsibilities, but other types of support need to be added to provide relief from distress experienced (theoretical proposition 2). Methodological triangulation has also been applied to illustrate how theoretical propositions can be produced by drawing on the findings from a Finnish study by (Fig. 5) . The purpose of this longitudinal study was to describe, explain and understand the subjective health related quality of life (QoL) and life course of people with coronary artery disease (CAD). A longitudinal quantitative study was undertaken during the Respite care provides relief from primary care providers' on-going care-responsibilities. For relief from distress experienced from caregiving other types of support need to be added. In the quantitative data, the majority of the carers showed a negative / no change in scores post respite stay. Yet most items were on emotional, not instrumental changes. Qualitative results showed that respite care was considered to be important to most of the carers as it enabled them to have a break and a rest from ongoing care-responsibilities. Respite care, provides relief from caregiving to primary care providers. Theoretical level Proposition 1 Proposition 2 Fig. 4 . Illustrating the use of triangulation (Erzberger and Kelle, 2003) on divergent results using the study by . year post treatment and 19 individuals also attended thematic interviews one year after treatment. This study is one of the few studies that clearly defines theoretical underpinnings for both the selected methods and their purpose, namely ''to obtain quantitatively abundant average information about the QoL of CAD patients and the changes in it as well as the patients' individual, unique experiences of their respective life situations'' (Lukkarinen, 2005:622) . The results of the quantitative analysis showed that the male and female CAD patients in the youngest age group had the poorest QoL. While patients' QoL improved in the dimensions of pain, energy and mobility it deteriorated on dimensions of social isolation, sleep and emotional reactions. From the viewpoint of methodological triangulation used in the study the aim of the quantitative approach was to observe changes in QoL at the group level and also explore correlations of background factors to QoL. The qualitative approach generated information concerning both QoL in the individuals' life situation and life course and the individuals' rehabilitation. Both the quantitative and the qualitative analysis showed the youngest CAD patients to have the poorest psychosocial QoL. The results obtained using qualitative methods explained the quantitative findings and offered new insight into the factors related to poor psychosocial QoL, which could be used to help develop theoretical propositions around these. Patients at risk of poorer QoL were those with an acute onset of illness at a young age that led to an unexpected termination of career, resulting in financial problems, and worries about family. This group also experienced lack of emotional support (especially the females with CAD) and were concerned for the illness that was not alleviated by treatment. The interviews and the method of phenomenological psychology therefore helped to gain insight into the participants' situational experience of QoL and life course, not detectable by the use of a questionnaire. Fig. 5 illustrates the theoretical propositions, empirical findings from qualitative and quantitative data and the relationships between these. The use of the mixed methods approach enabled a clearer understanding to emerge in relation to the lived experience of CAD patients and the factors that were related to poor QoL. This understanding allows new theoretical propositions about these issues to be developed and further explored, as depicted at the theoretical level. As the need for, and use of, mixed methods research continues to grow, the issue of quality within mixed methods studies is becoming increasingly important (O'Cathain et al., 2008 (O'Cathain et al., , 2007 . Similarly, the need for guidance on the analysis and integration of qualitative and quantitative data is a prominant issue (Bazeley, 2009 ). This paper firstly intended to review the types of analytic approaches (parallel, concurrent or sequential data analysis) that have been used in mixed methods studies within healthcare research. As identified in previous research (O'Cathain et al., 2008) , we found that the majority of studies included in our review employed parallel data analysis in which the different analyses are not compared or consolidated until the full analysis of both data sets have been completed. A trend to conduct separate analysis on quantitative and qualitative data is apparent in mixed methods healthcare studies, despite the fact that if the data were correlated, a more complete picture of a particular phenomenon may be produced (Onwuegbuzie and Teddlie, 2003) . If qualitative and quantitative data are not integrated during data collection or analysis, the findings may be integrated at the stage of interpretation and conclusion. Although little pragmatic guidance exists within the wider literature, Erzberger and Kelle (2003) have pub- Theoretical level The quantitative data showed that the male and female CAD patients in the younger age group had the poorest HRQoL in some dimensions over time. Qualitative results provided an explanation for the poorer psychosocial HRQoL obtained by the quant approach. Showing the crucial factor to be age related aspects of the life situation. Patients at risk for poorer HRQoL had acute onset of illness, at young age, unexpected termination of work, financial problems, worries about family and illness and experienced lack of emotional support, especially female CAD patients. (Erzberger and Kelle, 2003) to develop theory from the study by lished some practical advice, on the integration of mixed methods findings. For mixed methodologists, the 'triangulation metaphor' offers a framework to facilitate a description of the relationships between data sets and theoretical concepts and can also assist in the integration of qualitative and quantitative data (Erzberger and Kelle, 2003) . Yet despite the fact that the framework was published in 2003 within Tashakkori and Teddlie's (2003) seminal work, the Handbook for Mixed Methods in Social and Behavioural Research, our search revealed that it has received little application within the published body of work around mixed methods studies since its publication. This is surprising since mixed methodologists are acutely aware of the lack of guidance with regards to the pragmatics and practicalities of conducting mixed methods research (Bryman, 2006; Leech et al., 2010) . Furthermore, there have been frequent calls to move the field of mixed methods away from the ''should we or shouldn't we'' debate towards the practical application, analysis and integration of mixed methods and its' findings and what we can learn from each other's work and advice. Consequently, we have a state of ambiguity and instability in the field of mixed methods in which mixed methodologists find themselves lacking appropriate sources or evidence to draw upon with which to facilitate the future design, conduct and interpretation of mixed methods studies. It is for these reasons that we, in this paper, also intended to identify and select studies that could be used as examples for the application of Erzberger and Kelle's (2003) triangulation metaphor. When reviewing the studies it was clear that the majority of theoretical assumptions were implicit, rather than explicitly stated by authors. Wu and Volker (2009) previously acknowledged that while studies undoubtedly have a theoretical basis in their literature reviews and the nature of their research questions, they often fail to clearly articulate a particular theoretical framework. This is unfortunate as theory can help researchers to clarify their ideas and also help data collection, analysis and to improve the study's rigour (Wu and Volker, 2009 ). When using triangulation as a methodological metaphor (Erzberger and Kelle, 2003) , researchers are encouraged to articulate their theoretical propositions and to validate their conclusions in relation to the chosen theories. Theory can also guide researchers when defining outcome measures. Should the findings not support the chosen theory, as shown in our examples on complementary and divergent results, researchers can modify or expand their theory accordingly and new theory may be developed (Wu and Volker, 2009) . It is therefore our belief that using triangulation as a methodological metaphor in mixed methods research can also benefit the design of mixed method studies. Like other researchers (O'Cathain et al., 2008) , we have also found that most of the papers reviewed lacked clarity in whether the reported results primarily stemmed from qualitative or quantitative findings. Many of the papers were even less clear when discussing their results and the basis of their conclusions. The reporting of mixed methods studies is notoriously challenging, but clarity and transparency are, at the very least, crucial in such reports (O'Cathain, 2009) . Using triangulation as a methodological metaphor (Erzberger and Kelle, 2003) may be one way of addressing this lack of clarity by explicitly showing the types of data that researchers have based their interpretations on. It may even help address some of the issues raised in the debate on the feasibility of integrating research methods and results stemming from different epistemological and ontological assumptions and paradigms (Morgan, 2007; Sale et al., 2002) . In order to carry out methodological triangulation researchers also need to identify and observe the consistency and adequacy of the two methods, positivistic and phenomenological regarding the research questions, data collection, methods of analysis and conclusions. While we used systematic principles in our search for mixed methods studies in healthcare research, we cannot claim to have included all such studies. In many cases, reports of mixed methods studies are subjected to 'salami slicing' by researchers and hence the conduct of, and findings from, individual approaches are addressed in separate papers. Since these papers are often not indexed as a 'mixed method' study, they are undoubtedly more difficult to identify. Furthermore, different terminologies are used to describe and index mixed methods studies within the electronic databases (Halcomb and Andrew, 2009a) , making it challenging to be certain that all relevant studies were captured in this review. However, the studies included in this review should give a sufficient overview of the use of mixed analysis in healthcare research and most importantly, they enable us to make suggestions for the future design, conduct, interpretation and reporting of mixed methods studies. It is also important to emphasise that we have based our triangulation examples on the data published but have no further knowledge of the analysis and findings undertaken by the authors. The examples should thus be taken as examples and not alternative explanations or interpretations. Mixed methods research within healthcare remains an emerging field and its use is subject to much debate. It is therefore particularly important that researchers clearly describe their use of the approach and the conclusions made to improve transparency and quality within mixed methods research. The use of triangulation as a methodological metaphor (Erzberger and Kelle, 2003) can help researchers not only to present their theoretical propositions but also the origin of their results in an explicit way and to understand the links between theory, epistemology and methodology in relation to their topic area. Furthermore it has the potential to make valid inferences, challenge existing theoretical assumptions and to develop or create new ones. None declared. None. References (country of origin) Bureaucratic Itineraries in Colombia. A theoretical and methodological tool to assess managed-care health care systems Patient views on reminder letters for influenza vaccinations in an older primary care patient population: a mixed methods study Senegalese grandmothers promote improved maternal and child nutrition practices: the guardians of tradition are not averse to change Qualitative to quantitative: linked trajectory of method triangulation in a study on HIV/AIDS in Goa Use of outcomes information in acute inpatient rehabilitation Breaking it down': Patient-clinician communication and prenatal care among African American women of low and higher literacy Using mixed methods for evaluating an integrative approach to cancer care: a case study Development of a culturally specific instrument for mammography screening: an example with American Indian women in Vermont A study of clinical questions posed by hospital clinicians Exploring the hand hygiene competence of student nurses: a case of flawed self assessment The strengthening families programme (SFP) 10-14 and subscale misuse in Barnsley: the perspective of facilitators and families Risk reduction as an accepted framework for safer-sex promotion among women who have sex with women HPV knowledge among HPV+ women Increasing nurses' knowledge and skills for enhanced response to intimate partner violence Factors important for the measurement of social comparison in chronic illness: a mixed-methods study Exploring the quality of dying of patients with chronic obstructive pulmonary disease in the intensive care unit: a mixed methods study Included but isolated: early intervention programmes provision for children and families with chronic respiratory support needs Cardiovascular disease management: time to advance the practice nurse role? Positive futures? The impact of HIV infection on achieving health, wealth and future planning Initial evaluation of a computer-based medication management tool in a geriatric clinic Women's work and symptoms during midlife: Korean immigrant women A mixed methods study to investigate needs assessment for knee pain and disability: population and individual perspectives The association between young people's knowledge of sexually transmitted diseases and their behaviour: A mixed methods study Women's views on reminder letters for screening mammography An empirically derived taxonomy of factors affecting physicians' willingness to disclose medical errors Physician and staff perceptions of barriers to colorectal cancer screening in Appalachian Kentucky The relationship between patients' experiences of continuity of cancer care and health outcomes: a mixed methods study Identifying patient-relevant endpoints among individuals with schizophrenia: an application of patient-centered health technology assessment Preventing ventilator-associated pneumonia in the United States: a multicenter mixed-methods study The VA Ostomy Health-Related Quality of Life Study: objectives, methods, and patient sample Gaps in vaccine financing for underinsured children in the United States Formative research methods to understand patient and provider responses to heart attack symptoms Methodological triangulation showed the poorest quality of life in the youngest people following treatment of coronary artery disease: a longitudinal study Table 2 (Continued ) Using the PEN-3 model to assess knowledge, attitudes and beliefs about diabetes type 2 among Mexican American and Mexican Native men and women in North Texas The group matters: an explorative study of group cohesion and quality of life in cancer patients participating in physical exercise intervention during treatment Feasibility of a standardised quality of life questionnaire in a weekly diary format for inoperable lung cancer patients Street workers and internet escorts: contextual and psychosocial factors surrounding HIV risk behavior among men who engage in sex work with other men Hospital performance: competing or shared values? A participantory program evaluation of a sustem change program to improve access to infomration technology by people with disabilities Community management of intellectual disabilities in Pakistan: a mixed methods study Minority health and small numbers epidemiology: a case study of living conditions and the health of children in 5 foreign Roma camps in Italy Measuring wheelchair intervention outcomes: development of the wheelchair outcome measure Barriers and facilitators to the implementation of the collaborative method: reflections from a single site Decision tree model of the treatment-seeking behaviors among Korean cancer patients Barriers to pain management: caregiver perceptions and pain talk by hospice interdisciplinary teams Preliminary study of OCD and health disparities at the U.S.-boarder. Hispanic Health Care International Case series: mental health needs and perspectives of rural children reared by parents who abuse methamphetamine Patient choice: an influencgin factor on policyrelated research to decrease bedrail use as physical restrain Treatment-seeking delay among women with acute myocardial infarction: decision trajectories and their predictors Closing the gap? The effectiveness of referred access family support services If the price is right: vagueness and values clarification in contingent valuation Improving district nurses' confidence and knowledge in the principles and practice of palliative care People make assumptions about our communities': sexual health amongst teenagers from black and minority ethnic backgrounds in East london An exploration of family carers' experience of respite services in one specialist palliative care unit Barriers and facilitators influencing call center nurses' decision support for callers facing values-sensitive decisions: a mixed methods study A support intervention to promote health and coping among homeless youths Measuring key integration outcomes: a case study of a large urban health center Factors influencing teachers' implementation of an innovative tobacco prevention curriculum for multiethnic youth: Project SPLASH An exploratory mixed-methods study of Scottish midwives' understandings and perceptions of clinical near misses in maternity care Providers' perceptions of spinal cord injury pressure ulcer guidelines Balancing professional and personal satisfaction of nurse managers: current and future perspectives in a changing health care system Physical activity and mental well-being typologies in corporate employees: A Mixed methods approach Are older patients' cardiac rehabilitation needs being met Thai adolescent survivors 1 year after the 2004 tsunami: A mixed methods study A contrast between mothers' assessments of child malnutrition and physical anthropometry in rural Mexico: a mixed methods community study Design and development process of patient-centered computer-based support system for patients with schizophrenia spectrum psychosis Multimethod evaluation of health policy change: an application to Medicaid managed care in a rural state Caregiver grief in terminal illness and bereavement: a mixed-methods study A mixed methods study to compare models of spirometry delivery in primary care for patients at risk of COPD References (country of origin) Exploring help-seeking for ADHD symptoms: a mixed-methods approach Work-heart balance: the influence of biobehavioral variables on self-care among employees with heart failure Education for new role development: the Community Matron in England Physical activity experiences of boys with and without ADHD Mental health: patients' experiences of patient education during inpatient care Understanding adversity and resilience at older ages. Sociology of health & illness Factors impacting readmissions of older patients with heart failure Perceptions of the religion-health connection among African Americans in the southeastern United States: sex, age, and urban/rural differences Coping strategies and styles of family carers of persons with enduring mental illness: a mixed methods analysis Satisfied carers of persons with enduring mental illness: who and why? Organization development strategies for continuing medical education. The Journal of Continuing Education in the Mixed-methods exploration of parents' health information understanding Challenges to research in urban community health centers Innocent or culpable? Meanings that emergency department nurses ascribe to individual acts of violence Ties That Protect: An Ecological Perspective on Latino/a Urban Pre-Adolescent Drug Use Meaning and practice of palliative care for nursing home residents with dementia at end of life Perceptions about competing psychological problems and treatment priorities among older adults with depression Factors associated with the development of expertice in heart failure self-care Mental health triage nursing: an Australian perspective A mixed methods study on the needs of Korean families in the intensive care unit The reality of returning to work and training: experiences from a long-term unemployment project Evaluation of patient satisfaction with nursing care: quantitative or qualitative approach? Development of the specialist menopause pharmacist (SMP) role within a research framework Barriers to evidence-based practice in primary care nursing-why viewing decision-making as context is helpful Bureaucratic itineraries in Colombia. A theoretical and methodological tool to assess managed-care health care systems Mixed Methods Research for Nursing and the Health Sciences Use of outcomes information in acute inpatient rehabilitation Social influences on fertility: a comparative mixed methods study in Eastern and Western Germany Prologue: mixed methods for novice researchers: reflections and themes Integrating quantitative and qualitative research: how is it done? Barriers to integrating quantitative and qualitative research Exploring help-seeking for ADHD symptoms: a mixed-methods approach Development of a culturally specific instrument for mammography screening: an example with American Indian women in Vermont A study of clinical questions posed by hospital clinicians Research blenders: commentary and response. Transitions to new methodologies in nursing sciences Cochrane Handbook for Systematic Reviews of Interventions Research Design: Qualitative, Quantitative and Mixed Methods Approaches Designing and Conducting Mixed Methods Research Making inferences in mixed methods: The rules of integration The value of combining qualitative and quantitative approaches in nursing research by means of method triangulation Mixed Methods Research for Nursing and the Health Sciences Introduction to mixed methods research for nursing and the health sciences Cardiovascular disease management: time to advance the practice nurse role? Mixing qualitative and quantitative methods: triangulation in action Mixed methods research: a paradigm whose time has come Satisfied carers of persons with enduring mental illness: who and why? The International Sociological explanations between micro and macro and the integration of qualitative and quantitative methods Undertaking Systematic Reviews of Research on Effectiveness: CRD's Guidance for those Carrying Out or Commissioning Reviews. Centre for Review and Dissemination Mixed Methods Research for Nursing and the Health Sciences Evaluating mixed research studies: a mixed methods approach Methodological triangulation showed the poorest quality of life in the youngest people following treatment of coronary artery disease: a longitudinal study The group matters: an explorative study of group cohesion and quality of life in cancer patients participating in physical exercise intervention during treatment Paradigms lost and pragmatism regained: methodological implications of combining qualitative and quantitative methods Mixed Methods Research for Nursing and the Health Sciences Why, and how, mixed methods research is undertaken in health services research in England: a mixed methods study The quality of mixed methods studies in health services research A framework for analysing data in mixed methods research On becoming a pragmatic researcher: the importance of combining quantitative and qualitative methodologies Revisiting the quantitativequalitative debate: implications for mixed methods research Improving district nurses' confidence and knowledge in the principles and practice of palliative care An exploration of family carers' experience of respite services in one specialist palliative care unit Toward integrating qualitative and quantitative methods: an introduction Editorial: the new era of mixed methods Handbook of Mixed Methods in Social & Behavioural Research The use of theory in qualitative approaches to research: application in end-of-life studies Not required.