Meta-Synthesizing Qualitative Research in Information Systems

Hossana Twinomurinzi, School of Computing, University of South Africa. Email: twinoh@unisa.ac.za
Roy D. Johnson, Gordon Institute of Business Science, University of Pretoria. Email: roy@up.ac.za

INTRODUCTION

There is a continuing debate about the lack of Information Systems (IS) theory and an accompanying over-reliance on theories borrowed from other disciplines versus the assertion that there are indeed IS theories although some not well known (Straub, 2012). This paper offers a tool to assist in building IS theories at comprehensive (macro), mid-level (meso) and micro-levels. Comprehensive level theories attempt to bring together all the factors from different contexts into one all explanatory theory. Middle level theories target multiple factors that are particularly important to a domain. Micro-level theories focus on single-factors of a phenomenon within a domain (Ward and Hudson, 1998).

Theory is an important lens in all research and there are always tradeoffs in choosing what theory to adopt in a research project (Harris, 1975). Theories are adopted because they enable and guide a researcher to analyze, explain, predict and prescribe a phenomenon in its context (Barrett and Walsham, 2004; Gregor, 2006). Despite the importance of theory, there is a deep seated concern that there is no core IS theory (Weber, 2003), and that the means of developing theory in IS remains unclear. The lack of theory building tools in IS has resulted in ad hoc IS frameworks and little known IS native theories (Straub, 2012). Such theories and frameworks often ignore the contributions of previous findings (Ward and Hudson, 1998).

There is a need for new and contemporary IS theories rather than continuing to refine existing theories which are unlikely to fit the context of today (Kalmar and Sternberg, 1988; Webster, 2014). The research challenges of today are much more complex (e.g., poverty, globalization, human-trafficking) and require more than one disciplinary perspective to solve them (Galliers, 2011). The problems are interdependent, not isolated to particular sectors and are often not predictable (Klein and Huynh, 2004) calling for a multidisciplinary approaches.

The field of IS can learn a great deal from viewing problems in a world increasingly influenced by rapidly changing technology. The call for the creation of new IS theories does not in any way belittle the importance of theory refinement, rather, the call is for new theories that place value on adding a new perspective to the field of IS in particular.

It is wasteful to have such a rich set of qualitative IS findings and not be able to tie them into comprehensive, mid-level or micro-level theories. Community Informatics as an interdisciplinary area within IS leans a great deal on qualitative approaches to present the rich experiences of people within the community setting (JoCI, 2015). The full extent of ideas emanating from such deep studies are as such not fully developed into substantive theories that better explain the role of ICT and the internet in emancipating communities (Ward and Hudson, 1998). There is on the other hand a danger to creating theories for the sake of creating theories. A poor theory-development approach can result in a poor design, whereby constructs are thrown together in a theory without properly fitting them together (Kalmar and Sternberg, 1988).

The 21st Century has seen growth in the use and employment of qualitative methods as an accepted means of inquiry in IS. The growth of qualitative methods is evidenced by the rise in academic journals with special tracks or entire publications dedicated to qualitative methods. While the qualitative approach brings a much needed contextual perspective, one of the greatest contentions is that findings from qualitative research are often isolated and stand as irreconcilable islands of knowledge doomed to irrelevant speculation and to re-inventing of the wheel (Walsh and Downe, 2005). It is argued that the isolative aspect of qualitative methods makes such findings difficult to integrate into policy initiatives, to situate into larger programs, or even contribute to the improvement of other similar studies (Scruggs et al., 2006; Zimmer, 2006). Policies and wide-ranging programs are typically designed to be as broad and inclusive as possible.

Qualitative Meta-Synthesis (QMS) is a qualitative method of inquiry that uses as data the findings from other qualitative studies. QMS focuses on a particular phenomenon of interest, and infers relationships between the studies to create a common frame of reference about the phenomenon (Zimmer, 2006). The appeal of QMS arises from the process of integrating similar qualitative studies. A fuller understanding of the phenomenon thereby elevates the solitary findings to more conclusive stances (Thorne et al., 2004), creates higher levels of abstraction (Jensen and Allen, 1996) and leads to the development of theory (Estabrooks et al., 1994). The theories created from QMS are better able to inform policy and practice in preference to the isolated findings from the constituent qualitative studies. The development of higher levels of abstraction also appeals to the generalizability of qualitative findings in practice (Sandelowski et al., 1997). This paper presents the QMS as an approach and a means to bring together qualitative findings in IS.

The remainder of the paper is structured as follows. The history of the qualitative meta-synthesis approach is discussed and is followed by six guidelines for using the QMS method in IS. Two IS papers are examined to identify how QMS has been used in IS plus examining the strengths and weakness when QMS was applied. Finally, common misunderstandings of QMS are highlighted.

THE QUALITATIVE META-SYNTHESIS (QMS) APPROACH

Qualitative Meta-Synthesis (QMS) is informally ascribed to the authors of Grounded Theory, Glaser and Strauss (1967). In their syntheses of the process of life and major life transitions, QMS was described as an extension of Grounded Theory in Status Passage (Glaser and Strauss, 1971). QMS has since been described with different terms such as meta-study, meta-ethnography, grounded formal theory, qualitative meta-analysis and meta-synthesis in human sciences research. The 'meta' concept refers to the need to look back and reflect on what has transpired in a particular field to date.

QMS has predominantly been used in the evidence based fields of medicine and education. For example, a 1985 QMS study revealed a self-care paradox in nursing studies eventually leading to the creation of a model to guide self-care by nurses (Stern and Harris, 1985). A more recent QMS study of type 2 diabetes papers revealed the critical necessity of paying attention to the little discussed subjective issues in the self-management of type 2 diabetes (Gomersall et al., 2011). In education, QMS uncovered four themes about how academics perceive the integration of information literacy; collaboration, information literacy pedagogy, information literacy skills and knowledge (Derakhshan and Singh, 2011). QMS has also been applied in other fields such as management to create a better understanding in transformational leadership (Walsh and Downe, 2005). More recently, two IS studies in e-government maturity models adopted the QMS method and created more generalised models to understand e-government maturity (Lee, 2010; Siau and Long, 2005) to be compared later in this paper .

IMPORTANT MISUNDERSTANDINGS OF QUALITATIVE META-SYNTHESIS

QMS is at times confused with similar approaches that use findings from more than one constituent study as their data, such as: quantitative meta-analysis (QMA), secondary data analysis/critical literature review, metatriangulation or qualitative comparative analysis (QCA). Comparisons of these similar approaches to QMS follow.

QMS versus Quantitative Meta-Analysis (QMA)

There is a sharp contrast between QMS and its quantitative counterpart, the quantitative meta-analysis (QMA). QMA seeks to combine and summarise past quantitative research by using statistical methods (Card, 2011). The aim of QMA is to estimate the strength of the relationships between the constituent studies using statistical measures such as correlation coefficients and weighted averages. For example, Adam Mahmood et al. (2000) meta-analysed IT end-user satisfaction from 45 end-user studies published between 1986 and 1998. Adam Mahmood et al. (2000) found positive and significant support for the influence of nine variables on end-user IT satisfaction in divergent contexts. The QMA study does not take into account the contextual influences that may affect each of the satisfaction studies.

QMA seeks to increase certainty in cause and effect relations while QMS seeks to understand and explain the findings from a group of similar studies (Walsh and Downe, 2005). QMA aggregates data and reduces it to a single unit for greater certainty. And though QMS reduces uncertainty, it does not attempt to move towards absolute certainty. QMS does not seek to neutralize alternative experiences, but rather to reveal them as well as build procedures to reveal those alternative experiences (Thorne et al., 2004).

QMS versus Critical Literature Review/Secondary Data Analysis

QMS goes beyond a critical literature review or secondary data analysis to integrating the findings from relevant qualitative studies on a given phenomenon. The result is a common understanding and not a consensual worldview. QMS does not seek to justify the unnoticeable political and ideological dynamics innately embedded in previous research (Thorne et al., 2004). QMS is about understanding interpretations and not synthesizing the interpretations (Thorne et al., 2004).

As a method, QMS seeks to achieve more and not less, where differences are discovered in the findings they are retained and complexities are enlightened (Thorne et al., 2004). For example, Shen and Khalifa (2010) critically reviewed the literature on the worldwide popularity and rapid proliferation of social networking sites (SNS) to understand why people use SNS. They found, among other things, that social influences play a role in the usage of SNS. The study does not explain the differences in the studies that were investigated but rather used the Theory of Acceptance and Use of Technology (TAUT).

QMS versus Metatriangulation

Metatriangulation is a qualitative approach that similarly explores multiple constituent studies in an attempt to gain greater insight from an emergent theme or resultant theories (Lewis and Grimes, 1999). Although the steps of metatriangulation are similar to QMS the emphasis of metatriangulation is the relationships between the data in the constituent studies and the inferences drawn from the data because of the influence of a selected paradigm (Saunders et al., 2003). Hence, the substantive theory from metatriangulation emerges from the meanings that are uncovered by different paradigms. On the other hand, the substantive theory from QMS emerges from the relationships between the findings of the constituent studies. Metatriangulation when used reflexively (Saunders et al., 2003) can play an important role in defining the quality of QMS constituent studies and in helping understand why some constituent studies come up with different findings.

By using metatriangulation to understand power and IT, Jasperson et al. (2002) discovered how IT may be seen as a driver for organizational change or a process used by interested parties to affect the systems in place depending on the paradigm adopted. By using QMS on the other hand, Lee (2010) finds commonalities in the maturity models of e-government without having to understand how meaning was assigned to each of the findings of the constituent studies because of the paradigm that was adopted.

QMS versus QCA

The essence of qualitative comparative analysis (QCA) is based on the intrinsic human nature of comparison and the quantitative idealism for certainty in cause and effect relationships. QCA is also a technique that attempts to summarize the commonalities between similar cases as probable causes of a phenomenon (Rihoux and Ragin, 2009). In QCA, an attempt is made to control the contextual conditions as in experimental designs. The exception is how commonalities in the cases are treated as strong indicators of the cause of a phenomenon. The strength of the relationships between cause and effect are then measured using traditional quantitative approaches such as statistics. For example, Kohli and Melville (2009) compare successful IT innovators to understand the underlying nature for their success. QCA enabled them to find the greatest commonality as developing a specialized set of competencies.

QCA uses raw data while QMS uses secondary data as its source. Although QCA and QMA both attempt to define cause and effect relationships, QMS places its emphasis on describing the whole picture based on secondary data. QMS does not compare and contrast but rather seeks to integrate creatively (Sandelowski and Barroso, 2003). This makes QMS an amalgamation of the interpretations of primary data from other authors (Zimmer, 2006).

Concerns and Criticisms of Qualitative Meta-Synthesis

There are three overarching criticisms of QMS (Scruggs et al., 2006) which we challenge:

1. The nature of the qualitative approach is antithetical to being summed up, which QMS attempts to achieve,
2. QMS trivialises the diversity and significant differences of the qualitative methods employed in each constituent study, and
3. QMS is relatively new and therefore lacks evaluation criteria and generalizability.

We consider each of the three criticisms respectively and propose solutions from IS to overcome each criticism.

Qualitative Meta-Synthesis is Antithetical to the Qualitative Spirit

In integrating qualitative studies, there is the real risk of violating the fundamental spirit of the qualitative approach; that meaning is socially constructed based on the context (Zimmer, 2006). Qualitative research honours multiple perspectives rather than making an attempt at arriving at single truths (Thorne and Paterson, 2000). Critics of QMS question the extent to which a researcher is able to portray the subjective experience of the primary data from the constituent studies, yet the subjective experience is three times removed from the primary data. Given that the data of the constituent study are the first level of interpretation, findings become a second level and the QMS thereby evolves to the third level of findings. This would give QMS its own epistemological slant.

The criticism further highlights the importance in QMS of making explicit the process and the trail of how the interpretations emerge into a theory or framework (Sandelowski and Barroso, 2003). In this paper, we argue that QMS brings together the different perspectives of a phenomenon to give a broader and more holistic perspective. For example, the paper by Lee (2010) makes an effort to bring out the subjective experience of each of the constituent studies while at the same time preserving each of the meanings that arose in those studies.

QMS Trivialises other Qualitative Methods

Summarising qualitative findings runs the risk of destroying the integrity, scholarly intent and the epistemological commitments of the constituent studies (Sandelowski et al., 1997; Zimmer, 2006). The process of QMS disassociates the findings of the constituent studies from the context in which they were carried out but in so doing casts doubt on the merit of the results, as to whether the results retain the qualitative flavour within which the studies are philosophically situated (Zimmer, 2006). It is therefore questionable how qualitative approaches which do not have similarity in purpose can be synthesised. It is argued that different epistemological commitments will lead to the generation of different kinds of knowledge which then increases the challenge of theory formulation as prescribed in QMS (Jensen and Allen, 1996).

The scope and depth of the different qualitative approaches may also make the approaches less compatible. For example, phenomenological studies are narrow but deep and less agreeable for abstraction; grounded theories are broad in scope, not extremely descriptive and seek more generalisation; while ethnographic studies are very culturally specific (Zimmer, 2006).

To overcome the danger of incompatibility of qualitative approaches, we propose that QMS must appeal to the truth claims in interpretivism as follows:

• engage the methodological procedure with rigour,
• reveal the pragmatic value of interpretive accounts, and
• acknowledge that a correspondence exists in the intersubjective understanding and agreement between the interpretive account, the interpretations of others, and the phenomenon under investigation (Zimmer, 2006).

Nonetheless, some approaches have a degree of intersubjective understanding between them. For example, grounded theory and some schools of ethnography share epistemological underpinnings in their symbolic interactionism backgrounds (Lowenberg, 1993). Zimmer (2006) further recommends the use of the questioning style of hermeneutics (Gadamer, 1994) to unearth more meaning from the constituent studies thereby opening up more intuitive analysis beyond depending on the intersubjective understanding between different qualitative approaches. Unearthing more meaning is possible since the different qualitative approaches share a common process which acknowledges the inevitable influence of a researchers own situatedness (Zimmer, 2006). Grounded theory is about inductively generating theory from data (Glaser and Strauss, 1967), transcendental phenomenology with describing individual experiences of phenomena via the objective and epistemological stance of the researcher, and hermeneutic phenomenology on describing and understanding individual experience from the ontological perspective of being-in-the-world, that is of shared or taken-for-granted contexts and meanings (Denzin and Lincoln, 2005).

A simpler approach to counter the criticism of incompatibleness of qualitative approaches is to analyse those which use similar qualitative methods and have similar epistemological underpinnings (Zimmer, 2006).

Lack of Criteria in QMS

The relative newness of QMS means that some aspects of the method are not fully established and that its rigour is still questionable (Walsh and Downe, 2005). The risk is that QMS can be captured by novice researchers who want to avoid the hard work of collecting original data or interacting with human-beings (Thorne et al., 2004). QMS calls for a great emphasis on reflexivity (Walsh and Downe, 2005) and should therefore be carried out by those who have expertise in the area or a strong commitment to the area (Sandelowski and Barroso, 2003).

To overcome the lack of criteria, Jensen and Allen (1996) suggest using an important criteria of credibility. Credibility is regarded as faithfulness in handling data and remaining true to its source to the extent that the constituent studies are able to recognise their experiences from the meta-synthesis. Credibility can also be accomplished through peer-review (Paterson et al., 1998; Walsh and Downe, 2005) and by internally validating the substantive theory that emerges from the QMS using quotes from the constituent studies and externally validating the substantive theory against existing literature or theory (Jensen and Allen, 1994). For example, Lee (2010), attempts to establish credibility by reviewing some of the constituent studies to the resultant common frame of reference.

The criticisms of QMS theory not being generalizable (Scruggs et al., 2006) is a reductionist criticism which is countered in the appreciation that QMS is an ever-expanding exercise that constantly peels back the multi-layered contexts to reveal a phenomena not previously seen through the standalone constituent studies (Sandelowski et al., 1997).

QMS METHOD AS ADAPTED FOR IS

The approaches that have been used in QMS differ (Sandelowski and Barroso, 2003) but have the same fundamental essence (Thorne et al., 2004). In this paper, we have aggregated the different approaches while taking into account the nuances of the IS field and the above criticisms; the following proposed six guidelines are the result:

Guideline 1: Plan

A great deal of effort is required in any major research undertaking. Thought should be given to how the activities and important research needs will be met including funding, meeting facilities, travel and others. A research proposal that meets the requirements of research funding reviews will ensure that the minimum and necessary requirements have been addressed. There ought to be provision for contingency funds as such studies tend to take more time and effort than anticipated.

A project manager may also be considered necessary, otherwise the research leader will need to play this role. The following areas represent important aspects to consider in the planning phase:

1. Determine the IS field in which the problem is situated (Thorne et al., 2004).
2. Establish an IS research team that is committed to the phenomena under investigation (Paterson et al., 2001). Big teams come with collaboration challenges while small teams sometimes lack all the skills needed to complete the tasks. It is equally necessary to manage the group dynamics within the team otherwise interpersonal conflicts could threaten the entire research effort.
3. Establish the purpose and scope of the study (Paterson et al., 2001). Establishing the scope of the study determines how manageable the QMS exercise will be; the narrower the better, the wider the more difficult. Scope has an effect on the generalizability of the study (Walsh and Downe, 2005). It is more desirable to have a narrower and tighter aim at the beginning. For example, a set of papers produced over a period of time could be carefully selected from an IS conference series with a known preference for the qualitative method.
4. Develop the IS research question (Paterson et al., 2001).

Guideline 2: Inclusion Criteria to Retrieve and Assess Primary Research

The nature of the qualitative method means that some primary research material may be quite lengthy, and may only be found in book chapters or unpublished manuscripts. Back-tracking using references and appeals to known authorities has been shown to be helpful (Walsh and Downe, 2005). Defining what qualifies to be part of the research defines the degree of quality. Quality in qualitative studies is a fiercely contested issue because of the contextuality of each study. The following guidelines assist in defining quality and the choice of primary papers:

a. Record and make explicit the IS inclusion criteria which will be used to retrieve the primary research (Thorne et al., 2004). Papers which refute the claims of other studies must be allowed as part of the papers to be retrieved. Three methods to assess the primary research are offered (Sandelowski et al., 1997).

• Integrate the findings of a single researcher's multiple studies in a related IS field. The danger of this approach is that it could limit multiple viewpoints since only one researcher is considered.
• Synthesise studies by different investigators in a related IS field.
• Quantitatively summarise key elements across the qualitative IS studies. A superficial summary should be avoided.

It is necessary to acknowledge and choose one of the above methods and remain transparent by giving reasons and implications of the chosen approach (Walsh and Downe, 2005).

b. Rigorously appraise the primary research first for relevance to the research question and then for scholarly IS quality (Walsh and Downe, 2005). Some QMS researchers completely bypass the quality requirement opting only for relevance. To measure for rigour and relevance, a technique which proposes a set of qualifying criteria can be adopted. An example of qualifying criteria are the critical appraisal guidelines of Atkins and Sampson (2002) who propose a set of five criteria classes each having a set of qualifying questions that can be used to evaluate qualitative single case study research:

• The way of thinking (e.g., Is a credible argument given for why a case study is appropriate?),
• The way of working (e.g., Have the criteria for analysis been confirmed by an independent researcher?),
• The way of controlling (e.g., Are the criteria used to select the appropriate case and participants clearly described?),
• The way of supporting (e.g., Does the study describe an orderly process for the collection of data?) and
• The way of communicating (e.g., Are the aims and objectives of the study clearly stated?).

The rigorous appraisal process could start out with one reference and broaden the search as the retrieval process ensues, with the synthesist progressively selecting the materials to be analysed (Bates, 1989).

c. Make explicit the contexts of the constituent studies, the related assumptions behind each of the findings and their effect on the findings (Paterson et al., 2001).

d. Detail the event timelines placing the constituent studies in temporal relation to each other (Sandelowski and Barroso, 2003).

Guideline 3: Meta-Data-Analyse

Once the primary papers have been identified, most will have developed their own new metaphors and concepts that relate to the context of the primary research. It is necessary to translate such metaphors, ideas and concepts from one constituent study to the other so as to preserve the contextual signifiers from each constituent study while at the same time elevating the constituent studies into new meanings (Noblit and Hare, 1988). The translation is easier when the QMS is focused on similar IS topics. However, caution should be exercised when different methodologies have been adopted in the constituent studies.

Classify and make clear the constant targeted comparisons by deliberately searching for similarities and differences in the findings of the constituent studies to clarify the overlapping and defining metaphors used in each study. Explore each constituent study for imported concepts since some may import terms from outside the field. An understanding of imported concepts will depend on prior knowledge of a field (Sandelowski and Barroso, 2003; Thorne et al., 2004).

Three reflexive methods can be embraced for the translation:

1. Make explicit the original authors meaning of key themes, metaphors, phrases, ideas, patterns, concepts and relations using a table while preserving meaning as far as is possible (Paterson et al., 2001; Walsh and Downe, 2005). Many times these are descriptive rather than interpretive and may require going back to original sources which the studies cited.
2. Adopt a hermeneutic and dialectic approach (Zimmer, 2006) or any other data analysis method such as the grounded theory or meta-ethnographical approaches (Walsh and Downe, 2005).
3. Employ a multifaceted system for grouping data based on criteria such as method, sample, date of publication, and specific issues related to the phenomenon of interest (Paterson et al., 2001).

Guideline 4: Meta-Method

The choice of research method has an influence on the results of any study. Every method has an epistemological underpinning, which constrains what to consider and what to ignore. Boeije (2010) constructs the influence of qualitative research method along a continuum of data transformation (Table 1 and explained in Table 2).

Source: Boeije, 2010

It is necessary to rate and make explicit the degree of data transformation chosen in the constituent primary study. For example, how would findings elicited from grounded theory and phenomenology be compared because the data are vastly different (Noblit and Hare, 1988)? A reflexive translation process is needed to explicitly recognise the different methodologies prior to and during the analytic stage (Sandelowski et al., 1997).

It is therefore important to integrate rather than simply organise the findings by:

1. Comparing the research designs and making transparent their epistemological assumptions (Paterson et al., 2001),
2. Checking the relations between the constituent studies for dissonance or similarity, and through juxtaposition use the compare and contrast method. Attempts to force-fit to illustrate homogeneity should be avoided, it is better to comprehend how the links are reciprocal, complementary or conflicting and,
3. Ensuring a hermeneutic dialectic in the process to confirm the whole makes sense to the parts and vice-versa (Walsh and Downe, 2005).

Guideline 5: Metatheorise

At this point, the influence of theory in each of the constituent studies is considered. Theory in its purpose to analyse, explain, predict and prescribe research has its boundaries defined by the domain and the core problems in that domain (Gregor, 2006). Theory further has different levels of abstraction as some are comprehensive, middle-level and micro theories (Ward and Hudson, 1998). The structure, the process and the degree of persuasiveness (Kalmar and Sternberg, 1988) of each of the theories adopted in the primary studies should as such be reciprocally translated and synthesised based on the theory's concepts.

Translating integrates rather than compares the findings of one to another using metaphors and concepts that could be applied to both. Translating is sometimes straightforward and reciprocal and at other times may be conflicting or even overlapping. Overlaps and conflicts can signal another emergent category that has not previously been identified. Therefore, differences should not be ignored in pursuit of only similarities since, "deviant data may be the raw material for another perspective" (Walsh and Downe, 2005: 209) and the absence of deviance can be suspicious (Sandelowski and Barroso, 2003; Thorne et al., 2004). The deviance can be overcome by:

1. Analysing the constituent studies for the implication of their theoretical orientations to the topic, research question and for future theory development (Paterson et al., 2001) and
2. Aggregating the findings and determine the magnitude of the different abstractions of theory adopted in the primary study based on whether the theory is a comprehensive, middle-level and micro-level theory and on whether the theory is designed primarily to analyse, explain, predict or prescribe (Thorne et al., 2004, p. 1346)

Guideline 6: Meta-Synthesise - Deconstruction and Generalise

The final stage follows to dynamically and iteratively deconstruct current ideas about a phenomenon and come to a clearer, deeper, "more socially responsible" theoretical understanding (Paterson et al., 2001: 111) of the phenomenon. Deconstruction requires thinking, interpretation, creation, theorisation and deep reflection to transform the findings into new clusters of metaphors (Thorne et al., 2004).

The clusters of metaphors that begin to emerge form the core themes of the new comprehensive, mid-level or micro-level theory. This theory should be capable of:

• Reflecting the tensions between contradictory or alternative explanations if the tensions suggest a lack of congruence and
• Conveying explicitly the value of the theory to the greater whole, the parts of the research question and the field in which it is situated (Walsh and Downe, 2005).

QMS APPLIED IN INFORMATION SYSTEMS

QMS has twice been applied to IS in the field of e-government to create a better understanding of e-government maturity models by Siau and Long (2005) and Lee (2010). E-government is a growing area of research, which suffers from a plethora of frameworks and models aimed at specific contexts. Because of the diversity in perspectives, it was timely to reflect on what a common framework could contribute to the field of e-government. Both studies emerged with unique common frames of reference for e-government maturity.

In the next section, we offer a discussion using tables for a side-by-side comparison of the two papers, using the proposed QMS guidelines.

Plan

The narrowness of scope in each paper was helpful for comparison. Both papers sought to meta-synthesize the different existing maturity models of e-government in order to reach a common frame of reference. The two papers were written five years apart. It is not clear from either paper if a research team was established and it is questionable if a research team would have added any value (Table 3).

Inclusion Criteria to Retrieve and Assess Primary Studies

The inclusion criteria used by both papers contain the repositories searched, timeframe, quality and context of each constituent study chosen. All the papers reviewed by Siau and Long (2005), with the exception of one (Moon, 2002), were included in the metasynthesis by Lee (2010). Moon's (2002) model is an adaptation of Hiller and Belanger (2001). Lee (2010) included Siau and Long's (2005) model noting that Siau and Long's (2005) model was equivalent to the model by Gartner (Baum and Di Maio, 2000) with one dimension, e-Democracy, incongruently included (Table 4).

Context

Context is a fundamental principle of qualitative research without which a study may not qualify as being qualitative. There is a need for balance between making explicit the context and the amount of information that will be allowed in a single paper. The context can be represented by including the following five requirements of qualitative research (Denzin and Lincoln, 2005):

1. Define what is studied and where it is studied,
2. Identify the qualitative strategies of enquiry to be used,
3. Articulate the research tools for collecting and analyzing empirical data,
4. Associate how the research design connected to the context, and
5. Reflect on how the data enabled the researcher to speak to the problems of context and change.

Only Lee (2010) presents a degree of detail for some of the primary studies but not for all studies. Siau and Long (2005) only provided the purpose of the primary studies but not the context of any studies.

Relevance or Quality

Lee (2010) acknowledges his choice was not to assess the quality of the primary studies citing the newness of e-government as an IS field, implicitly opting only for relevance. Siau and Long (2005) do not assess the primary studies for their quality and opt only for relevance. It is not surprising that relevance is chosen over quality as IS seeks to make the basic and pure sciences relevant in the real world (Lee and Hubona, 2009). Nonetheless, quality is necessary to determine whether the primary study remained true to the philosophical underpinnings that govern the research traditions that were adopted. Further, it is necessary to make explicit the level of analytical quality employed in each of the primary studies as an important criterion of qualitative research.

Meta-Data-Analyse the Primary Studies

There is value in juxtaposing the studies in order to see explicitly and refer to how the research designs and terms are related to each other across the different primary studies. Techniques from hermeneutics and grounded theory are helpful in enabling the process of relating commonalities from text. None of the papers juxtaposed the research designs used in the primary studies.

With regards to the terms, since the meaning of a term is only related to the context from which the primary study was taken, it gives further justification to making explicit the context of each of the primary studies. Both papers adopted a table to juxtapose the terms used across the primary studies to derive common terms. Lee (2010) used an interpretive approach to relate the terms from the different papers. Siau and Long (2005) do not make explicit their approach to relating the metaphors but rather attempted to force-fit many terms into one another, eventually abandoning those terms, which did not conform. Siau and Long (2005) critiqued some of the findings as being too narrow and thereafter propose a means to amend the findings.

It is antithetical to the nature of QMS and qualitative studies to propose amendments to findings but rather to accept findings seeing that the QMS analyst is already at least three times removed from the primary data. It is also quite unreasonable to disregard the contextual and lived experience, which each primary study attempts to bring out (Table 5).

Meta-Method to Compare Method

Meta-method touches on the issue of pluralism in qualitative research where different qualitative approaches employ different epistemologies, philosophical traditions and practices. Some qualitative approaches are regarded as eclectic where all methods are regarded as equal and the choice is made based on what is available and at hand while others are regarded as pragmatic where the ontologies may be combined. Different analysis approaches yield different layers of interpretation, or different views of the same phenomenon.

None of the papers mentioned the research designs that were used in the primary studies (Table 6).

Meta-Theorise

The process of translating and synthesizing concepts into categories requires a great deal of creativity and reflexivity. It is necessary during the process not to yield to the temptation of discarding meaning from QMS studies in favor of 'better' expert approaches, as this would suppress deviance. Rather, the extent of deviance should be made explicit, an inherent feature in qualitative studies. Techniques from qualitative research designs can assist as in the steps of meta-data-analysis and meta-method.

Lee (2010) adopts content analysis to meta-theorize while Siau and Long (2005) are not explicit in the approach used but appear to force-fit terms into categories, abandoning some in preference for other terms which conform (Table 7).

Deconstruct and Generalise

The common frame of reference is able to reflect the dissonances and similarities between the primary studies as well as be able to give relative explanations as to why there are differences and similarities. Each primary study should be identifiable in the substantive theory.

Lee (2010) explains the new common frame of reference linking it to the different primary studies. The limitations as well as the areas for further research with regards to e-government as an IS field are made clear illustrating what could and could not be achieved by the analysis. Siau and Long (2005) briefly explain the new common frame of reference but they neither relate it back to the constituent studies nor do they illustrate the contribution to the IS field of e-government (Table 8).

CONCLUSIONS

IS has over the past years seen an appreciation of the qualitative approach, with an increasing number of IS conferences and journals including calls for publications that use qualitative methods. Qualitative approaches appreciate that context differs and are comfortable with an apparent dissonance of results. Unfortunately, the appreciation of dissonance means that the results of such qualitative studies often stand alone.

Community Informatics, as a field within IS, leans a great deal on qualitative research methods to reveal the often unreported rich and intricate insights about the role that ICT and the Internet can play in emancipating communities. Community Informatics is therefore exposed to the lack of collated substantive theories that better explain the field to a novice researcher and practitioners. Novice researchers and practitioners often have difficulty in appreciating dissonance, or understanding that divergent findings can be appreciated. A substantive theory that brings together the dissonant findings from Community Informatics would make a significant contribution to IS.

Qualitative Meta-Synthesis (QMS) seeks to overcome the isolationist tendency by bringing together the solitary and dissonant findings and inferring relationships between them to result in a common frame of reference about a phenomenon under investigation (Zimmer, 2006).

Such a common frame of reference has implications for both IS theory and IS practice. For IS theory, the common frame of reference can be the beginning of IS theories. For practice, the frame of reference can serve as the basis for policies that are relevant to divergent contexts. For example, a great deal of research has been done on ICT for Development yet the area is yet to produce a substantive IS theory or make a functional contribution to policy reformation.

In this paper, we presented QMS as a suitable approach that can bring together the various findings from qualitative studies in IS from which substantive IS theories can be built at comprehensive, mid-level and micro-levels. We revealed how QMS has not been rigorously applied in IS and as such has not produced the wealth of insights that are possible from qualitative studies. The paper considered how QMS has been applied in other fields similar to Community Informatics and looking at the unique nature of IS, developed a set of guidelines to be followed in IS. The QMS guidelines when rigorously applied do not violate the basic tenets of the qualitative paradigm. The meta-synthetic process can be carried out in the same hermeneutic spirit of the interpretive or critical paradigm. The theory(s) generated from QMS are more generalizable and allow for a greater contribution to IS theory and practice.

ACKNOWLEDGEMENTS

We acknowledge the invaluable feedback given by Carol Saunders, Cathy Urquhart, Shirley Gregor, Michael Barrett and Geoff Walsham in reviewing drafts of this paper.

REFERENCES