Review Article

Invoking interactive qualitative analysis as a methodology in statistics education research

Anisha Ananth, Suriamurthee Maistry

Received: 19 Nov. 2019; Accepted: 13 Aug. 2020; Published: 04 Nov. 2020

Copyright: © 2020. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

This study investigated the use of interactive qualitative analysis (IQA), as a research methodology, to develop an understanding of students’ experiences of learning statistics in a threshold concepts-enriched tutorial programme. Interactive qualitative analysis methodology offered a systematic, rigorous and accountable approach to conducting qualitative research. The participants constructed their own meaning of reality from their experiences of interacting with the phenomenon in context and, in its refutation of traditional qualitative norms of enquiry that casts the role of the researcher as the expert, IQA stands out – entrusting participants with data generation, analysis and interpretation. Participants’ reflections of their experiences of the phenomenon under study are classified according to variously identified emergent themes called ‘affinities’. Relationships between these affinities are extricated and characterised in a visual representation of the phenomenon called a systems influence diagram. Thus, the researcher’s role was purely facilitative, greatly limiting the potential for skewed power relations and bias which is often hazardous in qualitative research. The paramount value of this article was that it offered a practical methodological approach to using IQA in qualitative statistics education research, in particular, and mathematical sciences education research, in general. A summarised account of the main findings of the broader study was also presented.

Keywords: affinities; interactive; qualitative; analysis; systems; influence; diagram.

Introduction

In today’s data-driven technocratic society, data and statistics form the bedrock for informed decision making.^1,2 As such, almost all disciplines have incorporated an introductory statistics course into their academic programme structure. Students majoring in other disciplines often make for very reluctant learners of statistics. This may be attributable to statistics often being described as a difficult subject to learn, because of the abstract nature of some of its concepts, the distinct way of thinking and analysis of problems that it requires, or the way in which it is traditionally taught. These issues manifest in high failure and dropout rates, and in concerns about the quality of learning (and teaching) in introductory statistics courses.^3,4

This study was motivated by unanswered questions about statistics students’ learning at a South African university of technology. Observations imply that some students seem to struggle to grasp fundamental disciplinary concepts and practices. This holds back their learning and affects progression in the discipline. Even among those who have passed the introductory statistics course, some appear unable to transfer their knowledge to their specific research domain or to apply it meaningfully to interpret real-world events. In South Africa, features of this country’s higher education context may intensify the challenges faced by both students and teachers of statistics, with research reflecting concerns around poor academic performance and low throughput in introductory statistics classes across higher education institutions. This literature is dominated by quantitative studies determining the impact on performance because of students’ characteristics and prior academic attainment, behaviour and motivation, or educational interventions, in terms of pedagogical or teacher-focused interventions.^{5,6,7,8,9,10,11}

For students, learning statistics can be viewed as requiring a mastery of essential concepts and modelling techniques that are associated with developing a new way of thinking, as they move through introductory modules and application to individual research interests – that is, a form of ‘transnumerative-thinking’ whereby students possess the skill to make sense of different representations of data to make sense of the world around them and their place in it.¹² Students often experience this conceptual crossing or transformation as a difficult process, and many struggle to develop statistics understanding and acquire the necessary statistical ways of thinking and practice. Whilst this difficulty is widely acknowledged in statistics education research, its sources and the processes by which students reach conceptual mastery are diverse and are often characterised in the research literature as being either cognitive or affective in nature.¹³ Furthermore, in this view, disciplinary features interact with individual and contextual aspects in affecting the course of learning.

There is clearly a need to deepen the understanding of students’ learning and explore the potential ways in which it may be more effectively supported and facilitated, in introductory statistics in the South African context. Unanswered questions that emerge are broader than the focus of the existing body of quantitative research: contemplating the experiences and processes of students’ learning and the sources of difficulty they encounter implies a holistic and deep exploration of learning. Difficulty in itself has strong connotations of personal experiences and affective responses, which are not accounted for in the predominant quantitative approach¹⁴ and seem to require a more encompassing framing that can account for individual learners and their personal context in interaction with the discipline.^15,16

Little or no qualitative or conceptual enquiry has been undertaken, and the understanding of how students learn statistics in South African higher education is superficial. Questions around the processes and experiences of learning statistics, and the sources and nature of difficulty encountered by students, remain largely unexplored. Consequently, teachers’ knowledge of how to support or facilitate learning is not supported by qualitative research, specifically research findings arising from qualitative studies that investigate learners’ experiences of disciplinary learning.

Qualitative research aims at generating a deep understanding of a specific phenomenon under study by weaving together an ‘explicit rendering of the structure, order and broad patterns found among a group of participants’.¹⁷ Qualitative researchers focus on the socially constructed nature of reality and how social experience is created and given meaning.¹⁸ This type of research offers a holistic perspective of the phenomenon by understanding that the phenomenon is a complex system of dynamic inter-dependencies and that these complex, multi-layered systems often cannot be reduced to simplistic systems. This is because of the phenomenon being subjected to a multitude of idiosyncratic interpretations and perspectives.¹⁹ Thus, in the interest of authenticity, qualitative research entails an interpretive, naturalistic approach, wherein the phenomenon is observed in its natural setting and interpreted using ‘thick descriptions representing the complexity of situations’,²⁰ in the words of the individuals who are part of the phenomenon. Thus, this research approach makes meaning of the phenomenon from the standpoint of the individuals who are active participants of the phenomenon.^20,21

However, qualitative research methodology is beset with issues arising from its supposed lack of rigour,²² credibility, transferability, dependability, confirmability, validity and reliability – the qualitative research criteria advocated over the positivist research criteria of validity and reliability.²¹ Interactive qualitative analysis (IQA)²³ is a ground-breaking approach to qualitative research, offering a panacea to the issues of power relations and biases traditionally associated with the qualitative research approach.²⁴ Interactive qualitative analysis methodology supports the ‘constructs such as credibility, transferability and dependability, while highlighting the concepts of validity and reliability’.²³ Interactive qualitative analysis has built-in features promoting rigour in both data generation and analysis,^25,26 which are evident in the design of the research. The research design is: (1) public and non-idiosyncratic, (2) replicable within reasonable bounds and (3) not dependent (especially for analysis) on the nature of the elements themselves.²³ As such, with the IQA research design, the issues commonly associated with qualitative research such as researcher bias, reflexivity or trustworthiness are eliminated, by having a transparent audit trail of the steps followed according to rigorous, reliable and replicable rules.²⁶

This article offers a practical methodological approach to using IQA in a qualitative statistics education study exploring students’ experiences of learning statistics in a threshold concepts-enriched tutorial programme. The study participants were enrolled in an introductory statistics course – Business Statistics II (BSTS 201) – at the Durban University of Technology. The detailed description of the IQA methodological process used in the study and presented in this article could be applied to other studies in the areas of mathematical science research.

Literature review

Interactive qualitative analysis²³ is a systematic, rigorous, accountable framework for the analysis of qualitative data.²⁴ Winston²⁷ describes IQA as a systematic, protocol-driven research method that combines the quantitative rigour of data analysis with the qualitative depth of descriptive interviews. The method involves generating data through two phases – conducting a focus group session with participants (or constituents in IQA terminology) and semi-structured individual interviews of participants. ‘Constituents’ is the favoured term in IQA to reflect the ‘knowledge and power’ influence. Whilst noting the aptness of this term, the word ‘participant’ was chosen to be used interchangeably with ‘constituent’ because it reflects students’ participation in the tutorial programme over the semester and the focus group and individual interviews.

In the first phase – the focus group session (and the focus of this article) – participants write down their thoughts, feelings, reflections and experiences of the phenomenon under study, which they then classify according to various themes called ‘affinities’. The participants then suggest a system of influence that exists between these affinities, which results in a unified systems influence diagram (SID). The SID is a graphical presentation of the phenomenon under study. The IQA method is congruent with a social constructivist approach to data generation and analysis,²⁶ as it lends itself to the precept that knowledge (of the phenomenon) is socially constructed from human experiences. The participants construct their own meaning of reality from their experiences of interacting with the phenomenon in context and in its refutation of traditional qualitative norms of enquiry that casts the role of the researcher as the expert, IQA stands out, entrusting participants with data generation, analysis and interpretation. Thus, the researcher’s role is purely facilitative, greatly limiting the potential for skewed power relations and bias often hazardous issues in qualitative research.

Theoretical framework

In this section, the philosophical assumptions of IQA, as they informed the study’s method and design, are described. The epistemological and ontological stance of IQA is captioned as the ‘Beliefs and Values Redux’ of IQA and is presented as a series of points by Northcutt and McCoy,²³ which will be briefly explained, as it relates to this study, in the following paragraphs.

Ontological perspective

‘IQA presumes that knowledge and power are largely dependent, that power influences which knowledge is determined to be relevant and irrelevant, important and unimportant’.²³ One of the IQA criteria for constituent (participant) selection is the degree of power that the constituency has over the phenomenon. This assumption is reflected in this study by selecting constituents from among the participants in the threshold concepts-enriched tutorial group to participate in the IQA focus group. Because these students were inextricably linked to the phenomenon under study – experience of learning statistics in a threshold concepts-enriched tutorial programme – they had the power and knowledge to reflect on their experiences of learning statistics.

‘IQA presumes that the observer and the observed are dependent (or … interdependent)’.²³ Qualitative research methods that may adopt the notion that data collection and analysis are separate and distinct processes and only the researcher is capable of interpreting the data are eschewed by IQA methodology. Interactive qualitative analysis methods involve the participants in generating, analysing and interpreting the data, whereas the researcher’s role is that of facilitator. In this study, the participants generated data by reflecting on their experiences of learning statistics, in a threshold concepts-enriched tutorial programme during the focus group sessions and in the follow-up semi-structured individual interviews. Participants analysed the data by grouping their reflections, thoughts and experiences various themes (affinities). Participants interpreted these data by suggesting a system of influence between the affinities. An SID is the final, visual representation of the phenomenon under study ‘prepared according to the rigorous and replicable rules for the purpose of achieving complexity, simplicity, comprehensiveness and interpretability’.²³

‘The object of research in IQA is clearly reality in consciousness’.²³ The second criterion of IQA for constituent selection is that of distance from (or closeness to) the phenomenon. The participants in the tutorial programme were chosen because they were closest to the phenomenon and, therefore, had the experience and authority to represent and dissect their reality of learning statistics in the threshold concepts-enriched tutorial programme. Their group realities were initially socially constructed and expressed in the focus group sessions, followed by an elaboration on their individual realities in the individual interviews.

Epistemological perspective

‘IQA insists that both deduction and induction are necessary to the investigation of meaning’.²³ During the focus group session, participants were required to categorise the generated data into categories of meaning (affinities) using induction, then to refine and define these affinities using both induction and deduction and then to finally, deductively, identify relationships of influence among the affinities:

These three stages of data roduction/analysis – correspond to the three formal classes of analysis of coding: emergent, axial and theoretical … IQA contends that econtextualized descriptions are useful and possible as long as they are backed up or grounded by highly contextualised ones, and as long as the process by which the text was decontextualized is public, accessible and accountable.^{23(p. 17)}

This article endeavours to provide the reader with the broader context within which this research situates itself by weaving this study’s findings into the expansive quilt of statistics education research.²³ Interactive qualitative analysis allows the constituents to weave together an interpretive quilt of their reality of experience of the phenomenon.²⁶

‘IQA is clearly favourable to theory, both from the point of view of inducing theory and of testing it’.²³ The SID – being the final product of IQA – is a visual representation of a mind-map of the group’s reality of the phenomenon under study, rather than a model imposed by previous research findings or theorists.²³ The mental model produced by the participants in this study is their theory of their conscious reality of their lived experiences of learning statistics, in a threshold concepts-enriched tutorial programme.²³

Methodology

The first phase of the IQA process entails the selection of constituents to participate in the focus group session that results in the generation of a visual representation of the dynamics of the phenomenon. The second phase of the IQA process involves conducting semi-structured individual interviews with the constituents. The aim of these interviews is to further probe individual meanings of the affinities generated in the focus group session and provide an opportunity for constituents to theorise and rationalise their understanding about the relationship between affinities. The main processes of the first phase of the IQA process will be outlined below in relation to the study of students’ experiences of learning statistics, in a threshold concepts-enriched tutorial programme.

Identifying constituents (participants)

The first phase of IQA is the creation of the focus group. The task of the focus group participants is to represent their reality of the phenomenon in terms of its components of meaning and to propose their hypotheses of the relationships amongst these components.²⁸ This is in harmony with the guiding principle of social constructivism that meaning making is socially constructed. The IQA’s criteria for the selection of constituents are as follows: (1) extent of power over the phenomenon and (2) distance from, or the extent to which, the constituent experienced the phenomenon.²³ In keeping with these criteria, the constituents for this study’s focus group were 17 participants who voluntarily participated in the semester-long threshold concepts-enriched tutorial programme. The participants in the tutorial programme were chosen because they were closest to the phenomenon, and therefore had the experience and authority to represent and dissect their reality of learning statistics. Furthermore, these participants shared a collective, zoomed-in experience of the phenomenon.

The interactive qualitative analysis focus group: A collective reality

Focus groups are considered a key data collection method in qualitative research.²⁹ Northcutt and McCoy²³ recommend a 12- to 20-member participation in the focus group, and the IQA prescribed format does not centre on verbal discussion, so as to reduce the possibility of bias being introduced by domineering personalities during the coding phase. The IQA focus group data collection techniques assist participants ‘in articulating perceived relationships among these experiences to produce a theory in perception or a conceptual map’.²³ In this study, the participants in the focus group generated responses to issue statements (see Ananth³⁰) that would capture the essence of and shed light on the participants’ experiences of learning statistics in a threshold concepts-enriched tutorial programme. The participants then engaged in approximately 45 min of silent brainstorming, during which they recorded all their individual, spontaneous responses on index cards, writing one thought or experience per card using words, phrases, sentences or drawings. Once it was apparent that most participants had captured all their responses on index cards, these were collected and randomly affixed to the wall. In all, 217 responses were generated (see Ananth³⁰). The researcher’s role was strictly facilitative – guiding the group through the IQA procedures – being conscious of not imposing one’s interpretation of the emergent data or influencing data generation with one’s perceptions of the phenomenon of students’ learning experiences of statistics. The following paragraphs contain an outline of the IQA focus group procedure as it related to this study.

Generating and naming affinities

At the commencement of the session, the participants were presented with index cards and issue statements. The issue statements were designed to probe and elicit answers to the research questions. The first task for the focus group participants was silent brainstorming. Northcutt and McCoy²³ suggest using guided imagery to help participants relax and clear their minds for the session. A silent brainstorming session ensued, allowing participants to engage with the issue statements. Participants recorded all their responses to individual problem statements on index cards, writing one thought/experience per card using words, phrases, sentences or imagery. After the brainstorming session – approximately 45 min – the index cards were randomly taped along a wall.

The clarification of meaning stage followed: the facilitator read each card out loud to achieve clarity and consensus on the meaning of the card – this lay the foundation for constructing a shared reality in the group.³¹ Next, in the stage known as inductive coding,²³ participants were requested to silently organise the cards into groups of meaning or themes, known as affinities. The next stage was axial coding, a deductive process that required the participants (as a group) to decide on a name to give to each cluster of response (affinity naming) and to sort any cards that might have been miscategorised into the appropriate group. The concept of affinities is akin to cogs turning a wheel – a minor but necessary component in the workings of the system.

This entire session took approximately 4 h to complete. Although IQA calls for a single focus group session, in this study two sessions were scheduled, 5 days apart, which allowed the write-up of the affinity meanings – a consolidated description of the meaning of the affinity, grounded in specific responses or examples – after the first session, for the group to check and confirm at the second session.

The affinity write-ups as confirmed by the participants provided thick, detailed reflections of their experience with the phenomenon under study and are provided in Boxes 1–4.

The affinities described above may be considered as the system building blocks or components of the group’s shared reality of their experiences of learning statistics and participating in the threshold concepts-enriched tutorial programme. These affinities were used to draw up the affinity relationship table (ART), the next task that students had to complete in the follow-up focus group session. Participants had to fill in the ART to explain how they perceive these affinities to relate to each other. This task is completed through a deductive process, according to IQA protocol.

Identifying relationships among factors

This process entailed theoretical coding and was conducted in the second focus group session. Once the participants checked and confirmed the description of the affinities that were prepared, they were asked to analyse the nature of the relationships of influence between each of the affinities, which were recorded in a matrix called an ART. For any pair of affinities (A and B), participants had to decide whether: A influences B (A → B), B influences A (A ← B) or no relationship of influence existed. Participants were also asked to provide a specific example, in everyday language, or as an ‘if … then’ hypothesis, to indicate how they saw the directionality.

The ARTs were analysed at a group level to create a composite inter-relationship diagram (IRD) – a matrix of all the perceived relationships in the system²³ – aggregating the responses for each relationship pair in the completed ARTs from 17 participants.

Interactive qualitative analysis adopts the Pareto principle (named after Economist Vilfredo Pareto who developed the concept in the context of the distribution of income and wealth among the Italian population) to statistically determine which of the inter-relationships should be included in the IRD. The Pareto principle or 80/20 rule observes that 20% of the variables in a system will account for 80% of the total variation in outcomes in the system.²³

Northcutt and McCoy²³ encourage a parsimonious representation of the system (the Law of Parsimony or Occam’s Razor advocates that an explanation of a natural phenomenon should be portrayed with economical descriptions reflecting simple but evocative models) – the optimal number of relationships to be included in the model of the system is also ‘the fewest number of relationships … that represents the greatest amount of variation (for the sake of comprehensiveness and richness)’.²³

In applying this principle, the steps prescribed by IQA were followed. The process and its outcomes are displayed in Table 1.³⁰

The first step is to conduct a frequency tally – the ‘votes’ for each of the 12 affinity pair relationships were counted and then recorded in an Excel spreadsheet. In total, the participants cast 100 votes for relationships of influence. No participant voted for ‘no relationship’ between any of the affinity pairs. The relationships were sorted in descending order of votes (frequency) and then the cumulative frequency was calculated (columns 1–4 in Table 1).

As each additional affinity pair was considered, the cumulative percentage of total possible relationships was calculated (column 5). Each additional relationship added 1/12 or 8.33% of the total (thus, the first three relationships, cumulatively, accounted for 25% or 3/12 of all possible relationships, whereas all 12 relationships accounted for 100%). The sixth column shows the cumulative percentage of frequency, that is, of votes cast (based on the total of 100 votes) for the successive affinity pairs. Thus, the first relationship drew 17 of the 100 votes or 17%; the first four relationships accounted for 58% or 58/100 votes. In other words, the entries in this column reflected how much of the total variation in the system (= 100 votes) was accounted for by the cumulative relationship pairs. The final column, Power, was calculated as cumulative percentage frequency, less cumulative percentage relationships.

Interactive qualitative analysis applies the MinMax criterion in deciding which relationships should be included in the group IRD. MinMax is a statistics decision rule used in game theory to minimise loss in the face of a maximum loss scenario. In IQA, it is used to determine the minimum number of relationships needed to be analysed, which accounts for the maximum variation in the system.²³

The decision involves optimizing a trade-off between two criteria: The composite should account for maximizing variation in the system (cumulative percentage based upon frequency) while minimizing the number of relationships in the interest of parsimony (cumulative percentage based on relations).²³

In line with the Pareto principle, in this study, although the proportions were not exactly 80/20, it was still clear from Table 1 that a relatively smaller proportion of the relationships accounted for most of the variation – six relationships (50% of the total) account for 81% of the variation in the system. Power reached a maximum value at six relationships, which accounts for of the variation in the system. Therefore, six relationships would be a justifiable choice for inclusion in the group IRD because it was the optimal number in the sense of the MinMax criterion.

Constructing the inter-relationship diagram

During the process of ‘rationalizing the system’,²³ the first step is to display a summary of the optimal number of perceived relationships in the system in the form of a matrix called the IRD. The IRD is created by placing arrows in the table, thus showing the direction of influence (cause or effect) in an affinity pair relationship. Arrows point only left or up and each relationship is recorded twice in the IRD. For example, in Table 2,³⁰ in row one, the up arrow in the column headed 2 indicated that affinity 1 (tut group) influenced affinity 2 (journey of understanding). The same relationship was captured in the second entry for that affinity pair, in row two, where the left arrow in the column headed 1 showed that that affinity was influenced by affinity 1. In both instances, the arrows point away from 1 towards 2. (In a typical IRD constructed according to IQA protocol, the affinity names are omitted. The names are included here purely for explanatory purposes.) The six relationship pairs identified for inclusion using the MinMax criterion were recorded in Table 2.

The value of delta is used to designate the relative position of an affinity within the system. With reference to this study, affinities with positive Δs such as tut group and journey of understanding were relative drivers or causes in the system, whilst those with negative Δs such as emotions and personal journey were relative outcomes or effects in the system. Those relative positions might be classified further. A driver that had many ‘out’ arrows and no ‘in’ arrows might be called a primary driver (tut group). ‘Primary driver has a high positive delta and is a major cause that affects many other affinities but is not affected by others’.²³ A secondary driver is distinguished as having more ‘outs’ than ‘ins’ (journey of understanding). It had a relative cause or influence on affinities in the system. The primary outcome is characterised by a high negative delta resulting from many ‘ins’ but no ‘outs’ (emotions). Thus, the primary outcome was a significant outcome that was affected by many of the affinities in the system. A secondary outcome revealed a relative effect and was identified as having more ‘ins’ than ‘outs’ (personal journey). Often affinities might have equal numbers of ‘ins’ and ‘outs’ – these affinities are referred to as circulator or pivot in the final representation of the system. In this study, there were no affinities that fit this description; thus, there were no circulator/pivot in this system.

The tentative assignment of affinities in the system is shown in Table 4.³⁰

This activity is called theoretical coding and creates an extended reality for the group through further discourse. The goal of this stage is ‘to identify the skeleton of a “theory in perception”’.²³

Constructing the systems influence diagram

The information presented in the IRD is graphically represented in the form of the SID. The SID is a graphical presentation of the entire system of influences and outcomes – a visual system of affinities and relationships among them.²³

The cluttered systems influence diagram

In developing the SID, all the affinities are arranged according to the tentative SID assignment table in topological zones: primary drivers to the left and primary outcomes to the right. Secondary drivers and secondary outcomes should be placed in-between the primaries. In zones that contain more than one affinity, these affinities should be placed vertically in descending order of delta. Arrows are then drawn, as connections between each affinity pair, in the direction of the relationship as represented in the IRD. This version of the SID is referred to as the cluttered SID as it is ‘saturated’ with each link present in the IRD.²³

Figure 1 shows the cluttered SID, which is a vivid representation of the system dynamics. The cluttered SID is the visual culmination of the findings from the focus group.³⁰

FIGURE 1: Cluttered systems influence diagram.

Uncluttered systems influence diagram

The cluttered SID serves the purpose of offering a comprehensive and rich description of the system but lacks in the objective of parsimony of an SID.²³ In other words, the detailed comprehensiveness of the cluttered SID, saturated with the many links, can dilute the explanatory power of the SID as the system becomes ‘boggled down by the details of the relationships’.²³ To reconcile the ‘richness-parsimony dialectic’ of the SID,²³ IQA proposes the removal of redundant links, i.e. remove direct links which instead can be facilitated by mediating affinities. Redundant links are removed by considering affinities at the extreme left (highest Δ) and the extreme right (lowest Δ). If there are alternate paths between the two deltas other than the direct link, then that direct link can be removed. To illustrate the point, in the cluttered SID for this study, link 1→3 would be considered redundant because 1→2 and 2→3, and thus the direct 1→3 link could be removed.

By eliminating the redundant links from the cluttered SID, the system is said to be ‘rationalized’,²³ thus yielding the uncluttered SID: ‘a simpler, more interpretable mental model – one that has optimum explanatory power’.²³

The uncluttered SID representing the participants’ experiences of learning statistics in a threshold concepts-enriched tutorial programme is presented in Figure 2.³⁰

FIGURE 2: Uncluttered systems influence diagram.

Because of its simplicity, the uncluttered SID paradoxically has the most explanatory power offering a visual composite mind map or mental model of the participants’ experiences in its rationalised form.²³

The visual representation may be expressed through language as follows.

The dynamics of participating in the tut group drove participants’ learning in statistics by influencing their journey of understanding in the discipline. Their disciplinary journey of understanding brought about a shift in the way students think about statistics and in the way they think about learning, in general. This shift in thinking about learning impacted on students’ personal journey, which culminated in an influence on their emotions.

Feedback loops, zooming and naming

A feedback loop comprises a system of at least three affinities wherein each affinity influences the other either directly or indirectly. For example, consider an arbitrary system, the links (A→B; B→C; C→A) comprises a feedback loop in which each of the three affinities, A, B and C, influences each other. Thus, in a feedback loop, the distinction between drivers and outcomes is blurred. A, B and C have individual, independent meanings but through their interconnectedness they have a collective meaning as a dynamic set of affinities.

Zooming refers to naming feedback loops with a name that succinctly captures the essence of the individual components comprising the feedback loop. By zooming out, simpler views of the SID are constructed. Northcutt and McCoy²³ explain that ‘zooming is actually just another form of affinity analysis conducted by the investigator for interpretive purposes’.

In this study, the SID featured no feedback loops. The uncluttered SID featured a simple, although evocative, linear system with no branching. However, the sequential interaction of tut group, journey of understanding and personal journey evoked connotations of a spiritual journey or pilgrimage of sorts. It was spiritual in the sense that participation in the tut group impacted on students’ understanding in the discipline (journey of the mind), which ultimately had bearing on students’ personal journey of self-discovery (spiritual journey). This pilgrimage captured the essence of a mind–soul connection – students initially embarked on a cognitive journey of the mind by participating in a threshold concepts-enriched tut group (a pedagogical tool) and ended in a personal journey of enlightenment and awareness of oneself. Thus, the sequential interaction between tut group, journey of understanding and personal journey was named ‘a pedagogical pilgrimage’ (Figure 3).

FIGURE 3: A pedagogical pilgrimage.

This minimised linear view of the system cannot be simplified further and comprises only two elements, ‘the bare minimum for a system’.²³ Students’ pedagogical pilgrimage ultimately impacts their emotions.

Interactive qualitative analysis interviews – Individual meaning making

The second phase of IQA entails conducting semi-structured individual interviews that serve a twofold purpose²³:

• Axial interview is an open-ended design to capture rich descriptions of the affinities from the participants, and the interviewer (researcher) probes the participants’ meaning of the affinities.
• The structured, theoretical interview design probes participants’ meanings of relationships between affinities.

The interview protocol is only drafted after the focus group and ART phase, because participants are expected to elaborate on their interpretation of the meaning of the affinities and the relationships between them. Whereas the focus group dealt with the generation of data that was socially constructed, representing the participants’ shared experience of the reality presented by the issue statements,²³ the individual interview stage created an opportunity to explore ‘individual differences in meanings vis-à-vis the issue’.²³ The composite interview mind-maps (the outcome of the individual interview stage) served as a source of data triangulation in the sense that if the participant ‘is representative of the constituency from which the group SID was derived, then the composite of all such interviews should be similar to the focus group SID’.²³ Interactive qualitative analysis relies on composite quotes, woven together from individual sources to sound like a single voice, to elaborate the affinities. This practice presents a trade-off, where, whilst a holistic and nuanced picture is created for the group, individual voices are not reflected or tracked in these aggregated quotes.

Findings

The focus group identified four affinities – themes or components of meaning of their learning in the tutorial programme – and the IQA processes led to the construction of the SID above, which captures how the group theorised the inter-relationships among these affinities. The features of the SID as a whole suggest some insights regarding the group’s understanding of their learning, which were substantiated by the thicker descriptive data from the reflective writings of the focus group brainstorming session, interviews and written journal reflections. The primary driver of the system was tut group, which impacted on every other affinity. The journey of understanding was a secondary driver and personal journey was a secondary outcome, whilst emotions was analysed to be a primary outcome.

Tut group – the group interactions and processes arising through the multiplicity of pedagogical approaches adopted (small group cooperative learning, activities involving the analysis of real data, keeping of reflective journals, being provided with the solution to tasks, and instructor immediacy) in the tutorial programme – enabled and supported cognitive, metacognitive and affective aspects of learning. Through processes of discussion and articulation within the group, students constructed an understanding of statistics concepts. They recognised this understanding based on ‘knowing why’ as being quite different to the mechanistic analysis they had tended to resort to in response to traditional lectured delivery. However, the progression in their understanding was difficult, and impasse was commonplace, where their primary source seemed to be in the abstract nature of many statistics concepts. Seeing their use through relatable applications facilitated understanding, and enhanced agency as students felt they were gaining worthwhile, empowering knowledge. Comprehending particular concepts and techniques was often experienced as a breakthrough that enabled the understanding of related ideas.

Cumulatively, these understandings constituted the development of students’ journey of understanding – a statistics point of view, which afforded them a changed perspective on real-world events. Their changing conceptions of knowledge were reflected in deeper approaches to learning and were accompanied by a sense of empowerment and capability, where before, many had felt self-doubt. These new conceptions of knowledge and learning, and of themselves as capable learners, were important metacognitive shifts that made up personal journey.

Emotions – affective responses, both positive and negative, emanated from learning and engaging with content, from assessment, and from the learning environment. Persisting and progressing in learning made demands on students’ psychological resources, such as self-awareness, self-belief, hope and resilience. Tut group had a significant influence on emotions: the sense of belonging, comfort and safety among peers and a sense of the teacher’s immediacy in the tutorial group encouraged students to express themselves freely, and for many, facilitated the emergence of a clearer sense of self in relation to the discipline. Student’s predominant feelings and emotions of their experiences of learning in statistics was one of the senses of achievement and accomplishment that translated into joy and love for the subject.

These summarised key findings of the larger study³⁰ highlight that learning has strongly affective aspects entwined with the cognitive; that it might entail periods of stuckness and liminality³²; that particular concepts are likely to be both troublesome and – once mastered – transformative³³ and that disciplinary learning has implications for students’ worldview and identity. Additional insights are suggested by participants’ portrayal of the influential role of tut group in their learning. Peers, teacher immediacy and the blending of multiple learning processes appear to hold significant potential for supporting cognitive, metacognitive and affective aspects of learning, and facilitating the emergence of a new sense of self in relation to the discipline.

Adapting Northcutt and McCoy’s interactive qualitative analysis framework for the South African higher education context

Interactive qualitative analysis is a novel method, which, as far as could be determined, has not been used in the extant statistics education research literature. In South Africa, IQA has been applied in the fields of educational psychology.^24,25 In a context similar to that of the current study, in students’ learning of economic threshold concepts, Goebel³¹ draws a comparison with the IQA methodology and the ‘transactional curriculum inquiry’ feature of the threshold concepts framework. This is done because in IQA methodology, researcher and students work together iteratively to explore the dynamics of the phenomenon, which is similar to the collective efforts of academics, students and curriculum developers in threshold concepts research.³⁴ Issues of trustworthiness, dependability and confirmability are significantly reduced in the IQA methodology,²⁵ as the participants are solely responsible for generating, analysing and interpreting the data.

The written reflections requested of participants were an addition to standard IQA practice, wherein it was hoped that it might offset some of the concerns around silence or conflict noted above, as a less focused and more private medium, reflective writing, might be a channel for insights students would not mention in the focus groups or interviews. Confidence in the affinities was increased when it was found that the reflective writing could be accounted for within those categories of meaning. This modification to IQA may also have enhanced the quality of the data by predisposing the participants to think more widely or introspectively during the focus group phase, because they would have reflected regularly on their learning over the preceding weeks.

Limitations of interactive qualitative analysis methodology as experienced in this study

The use of IQA revealed some limitations and possible hazards, as well as the advantages noted in describing the approach above. Many of the concerns relate to the pivotal role of the focus group, on which the rest of the approach may be seen to depend. Firstly, the communal nature of the focus group and its emphasis on group reality seem to reflect an assumption that there exists a single, albeit complex, group reality that can be satisfactorily captured in the SID. The IQA guidelines²³ do not offer steps to follow if participants cannot reach complete agreement before time and energy run out. In the focus groups, the researcher’s observations suggested that the participants reached consensus relatively smoothly, and therefore did not have to respond to this type of dilemma. A concern remains, however, that in seeking consensus, the focus group processes might simply be masking conflict and attaining compromise – the appearance of agreement. The silent nature of brainstorming and clustering in IQA, whilst offering advantages, may not reveal unresolved conflicts of meaning among participants.

The interviews may address some of the concerns around consensus and silence, by offering an opportunity to explore individual variations in the meaning of affinities. However, the interviews are themselves bounded and structured by the affinities, which inform the interview protocol and therefore largely determine the issues to be discussed. Thus, whilst the grounded, participant-driven nature of affinities (and their inter-relationships as depicted in the SID) is one of the IQA’s key strengths, this may be a double-edged sword: once confirmed, the affinities are set, and the researcher and readers are called on to trust that they are a fair representation of the participants’ reality. Whilst Northcutt and McCoy²³ point out that the interviews can act as a check of the affinities and influences, they do not offer comment on the possibility that the affinities and system produced by the focus group in the first place may be flawed. Again, this underlines the critical importance of the execution and functioning of the focus group.

Although IQA calls for a single focus group session, in this study had two scheduled sessions, 5 days apart, which allowed the time to write up the affinity meanings (as described above) after the first session, for the group to check and confirm at the second. Their agreement would subsequently affect the SID, the interviews and ultimately the findings of the study. In short, IQA protocol is not self-driving: the focus group phase needs careful planning and management, because all the phases that follow hang on the credibility of the affinities. Practical considerations should not be overlooked, given the importance of this phase. The focus group sessions were found to be more time-consuming than anticipated, and at times logistically challenging, with 17 participants ranging back and forth along the array of cards attached to a wall. Allowing for two consecutive sessions might be an advisable adjustment to the IQA prescriptions, although it also increases the demands made on participants.

It is believed that the willingness of the participants in this study to commit so much of their time to the focus group sessions and subsequent individual interviews was because of several factors, including the relationship established between the researcher with the group over the semester; their relationships with one another, their understanding and support of the purpose of the research; an appreciation of having their views heard and last, but certainly not least, the general improvement in understanding of content knowledge that they experienced in their working through of the tutorial activities. The level of commitment of the participants was essential to obtaining the depth of data that were generated in these phases of IQA.

Discussion

Together, the IQA processes of: (1) focus group and (2) semi-structured individual interviews enabled the generation of socially constructed descriptions of students’ experiences of learning statistics. The affinities (the elements of meaning of the phenomenon) and their perceived cause-and-effect interactions were grounded in the thick, rich descriptions in the words of the participants.

In this manner, IQA methodology attempts to reveal ‘truth’ as constructed by an individual or group by deliberately incorporating concepts from the following three important understandings of the meaning or theories of truth: correspondence theory of truth (CTT), coherence theory of truth (CoTT) and constructive theory of truth.²³

The CTT interprets truth as a correspondence with facts or reality as described through participants’ experiences. In this study, findings corresponded with participants’ lived reality of their experiences in learning statistics in the threshold concepts-enriched tutorial programme, as observed through the data sources.^26,31

The CoTT sees truth as being consistent with other true statements. The CoTT has three criteria:

• Structural coherence – coherence between ‘elements’ and ‘relationships’ that make-up a meaningful structure. In this study, this interpretation of truth would be reflected among the affinities and affinity relationships.
• Referential coherence – the study fits into a larger system with other studies – such as broader theoretical perspectives in this case.
• Dramatic coherence – where the characters are relatable and resonate with our own experience. This may be construed as participants’ individual meanings/interpretations of affinities and relationships of influence amongst affinities.

The constructivist theory of truth conveys the value of truth as being ‘useful’ or ‘pragmatic’ and that the ‘real’ truth of a proposition ‘lies in its potential for solving a problem’.²³ This study’s findings offer vivid descriptions of the phenomenon – insights into students’ learning in statistics – with implications for practical interventions in the statistics classroom and potential for further research.

This article set out to offer an exposition of IQA²³ and an example of its practical application. It drew attention to its particular merits, especially as it relates to affording voice and participation of research participants. Interactive qualitative analysis has special appeal in that it proposes a highly systematic research protocol that offers a very clear audit trail, as it relates to data collection, analysis and the development of research themes or findings in a study. As such issues of rigour, validity and reliability become transparent, making the assertions and insights gained more credible.

Acknowledgements

The authors have declared that no competing interest exists.

Both authors contributed equally to this work.

Ethical approval to conduct the study was obtained from the Humanities and Social Sciences Research Ethics Committee at the University of KwaZulu-Natal (Ethical Clearance Number: HSS/1057/017D, 17 July 2017).

This research article is based on the research findings supported in part by the Teaching Development Grant National Collaborative Project – Strengthening academic staff development in mathematical and statistical sciences in South Africa (Reference no: APP-TDG-037).

Data remain the intellectual property of UKZN but are available on request.

The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position of any affiliated agency of the author.

References