Objectives: This research aimed to determine which CSFs are the most important in the financial services sector of South Africa.

Method: The authors used a Delphi-technique approach with key project stakeholders in three BI projects in different business units of a leading South African financial services group.

Results: Authors regarded CSF categories of ‘committed management support and champion’, ‘business vision’, ‘user involvement’ and ‘data quality’ as the most critical for BI success.

Conclusions: Researchers in the BI field should note that the ranking of CSFs in this study only correlate partially with those a European study uncovered. However, the five factors the authors postulated in their theoretical framework ranked in the seven highest CSFs. Therefore, they provide a very strong validation of the framework. Research in other industries and other emerging economies may discover similar differences and partial similarities. Of special interest would be the degree of correlation between this study and future, and similar emerging market studies. Practitioners, especially BI project managers, would do well to check that they address the CSFs the authors uncovered before undertaking BI projects.

This research aimed to determine which CSFs are the most important in the financial services sector of South Africa. The research used a Delphi-technique approach to key project stakeholders in three BI projects in different business units of a leading South African financial services group.

The findings show that the CSF categories of ‘committed management support and champion’, ‘business vision’, ‘user involvement’ and ‘data quality’ are the most important in the participating business units. The top CSFs align almost perfectly with the five factors hypothesised in the much more parsimonious theoretical framework. The CSF rankings also correlate partially with those a European study generated.

In today’s ever-changing world of business, organisations need to be competitive and innovative in order to provide value to shareholders (Blenkhorn & Fleisher 2007). One way organisations can achieve this is to extract the maximum possible value from their internal data assets by using techniques like interactive graphical data analysis, data mining and predictive analytics. These techniques and tools are part of a discipline referred to as business intelligence (Hawking & Sellitto 2010).

However, the success BI implementations have achieved has varied across organisations and industries. Whilst each organisation and industry provides a specific context, researchers have identified a group of more generic factors critical to the success of BI projects. These factors, referred to as Critical Success Factors, cover a wide spectrum of influences like top manager support, market dynamics, data quality of the source systems and BI system usage (Adamala & Cidrin 2011). This research aimed to ascertain which CSFs organisations regard as the most critical in the South African financial services context.

Several recent studies have investigated the outcomes of organisational BI projects by using the CSF approach (Yeoh, Gao & Koronios 2007; Hawking & Sellitto 2010; Yeoh & Koronios 2010; Olbrich, Pöppelbuß & Niehaves 2012; Presthus, Ghinea & Utvik 2012).

Therefore, the research questions are:

• Which CSFs are the most critical in the South African financial services context?
• How do they compare against the actors hypothesised in the theoretical framework (see infra)?
• How do the CSF rankings correlate with those from a European study?

In line with the theoretical framework, the authors will rate CSFs in two contexts: project specific versus CSFs that are more general. They will compare the ratings for the individual projects to the overall general ratings to determine whether a generalisable set of CSFs applies across the projects. Because there seem to be no published studies on CSFs in BI implementations in South Africa, the authors hope that this research will be a platform for further research in this area.

The proper use of BI can have a material benefit on organisations’ bottom line (Ranjan 2008). Research that has shown the importance of BI for information technology (IT) executives has substantiated this (Luftman & Ben-Zvi 2010). Gartner highlighted the importance of investing in BI in 2009 when Gartner predicted that ‘BI system vendor revenue will reach $7.7 billion by 2012’ (Hawking & Sellitto 2010). However, the demand for BI, analytics and performance management software resulted in revenue that exceeded the 2009 prediction, coming in at $12.2 billion in 2011. This was a 16.4% increase over the 2010 revenue of $10.5 billion (Gartner Research 2012).

Organisations do not always recognise the benefits of BI (O’Brien & Kok 2006). The literature review by Hawkins (2010) validated this. Hawkins (2010) found that organisations do not realise the main driver of using BI, a higher quality of decisions. Business intelligence systems are complex entities and, with complexity, comes cost (Yeoh & Koronios 2010). Organisations need to manage, measure and justify expenditure to ensure that the information it yields meets the requirements they define (Lönnqvist & Pirttimäki 2006).

Depending on the industry and type of organisation, some factors will have a greater influence on the BI solution than others will. The challenge for organisations is to identify the factors that have the greatest influence on their BI system. An important criterion organisations should remember when they select the factors, is that they must have some effect on the factors for the duration of the project. The effect can be partial because they will set guidelines for how they expect to meet the target for a factor, whereas full control defines the expectations of what the target for the factor is precisely (Olbrich et al. 2012). By focusing on these factors, organisations will be able to provide the platform for increasing the potential success of the IT solutions.

Over the last five years, there have been several empirical studies on CSFs in BI (Yeoh et al. 2007; Hawking & Sellitto 2010; Yeoh & Koronios 2010; Olbrich et al. 2012; Presthus et al. 2012). In these studies, committed top manager support, source system data quality and user involvement emerged consistently as the most important CSFs.

An empirical case study research project identified the importance of a value driven framework for BI (Smith & Crossland 2008). It showed that aligning BI strategy with organisational strategy and explicitly measuring business benefits using metrics are key to realising the value of BI.

Pellissier & Kruger (2011) conducted a more recent empirical study in the long-term insurance industry. However, they focused only on a sub-set of BI: the use of strategic intelligence. Their research also found a lack of awareness and under use of intelligence capabilities. They recommended the use of a strategic intelligence framework to drive the intelligence processes to manage the complexity and reap the full benefits of strategic intelligence that improved decision-making, competitive advantage and innovation.

Ponelis (2011) conducted an exploratory study into using BI in knowledge-based small, medium and micro-enterprises (SMMEs) located in Gauteng. The research findings identified a number of CSFs that the SMMEs were not meeting. All the SMMEs had standard operational payroll software. However, only one had bought the BI add-on module and even that SMME was not using the BI capability optimally. Therefore, these SMMEs were accumulating data but were not converting it into information through BI technology because the owners (top managers) did not see the benefits of BI. Where they did have BI technology installed, there was a lack of skills to take advantage of its functionality. Reluctance to use consultants because of the perceived lack of knowledge transfer further compounded the problems. Interestingly, the earlier study by O’Brien & Kock (2006) also uncovered most of these issues and largely confirmed the findings.

On the other hand, there have been several successful achievements in the rollout of BI in the public sector. Awards to the South African Revenue Service (SARS) and the City of Cape Town acknowledged these. The Western Cape Education Department (WECD) has achieved further successes (Lutu & Meyer 2008). Researchers have identified that BI has an important contributory role to play in addressing the current service delivery needs in the public sector through the performance management system (PMS) (Hartley & Seymour 2011). The public sector would achieve this through understanding where it needs investments most and by monitoring the consumption of state resources.

One can identify CSFs at a granular level, depending on the context of the subject matter. Hartley & Seymour (2011) have researched the factors that affect the adoption of BI technology in the public sector. The researchers used the Heeks Design-Reality Gap model, an information systems development model for developing countries, to assess seven factors.

The model works on the principle that the closer the design requirement is to the reality (as is), the smaller the gap to adoption will be and, therefore, the higher the level of success that organisations can achieve. The results of the study showed, that for the BI system to provide valuable information, the quality of the data at source is fundamental and reporting must be timely. The South African Revenue Service, whose knowledge of the taxpayer has resulted in its increased ability to collect taxes, has demonstrated this. Another outcome was that, to build the data source accurately, organisations must identify and monitor the key processes. The data produces information that can help to identify and monitor service delivery needs.

A study by Bijker (2010) noted the importance of taking an incremental approach to BI to deliver sustained value to organisations. One of the business units that participated in this research, and which has delivered incremental benefits over a four-year period, has reinforced this finding.

However, the focus of this research is on measuring the contextual variables (the non-project specific organisational implementation factors). Therefore, the authors excluded team skills, source systems and development technology as factors. The focus was also on organisational and project implementation success, not technical success, because too many organisations easily consider BI projects ‘technical successes’ but do not use them. Therefore, only the top part of the Wixom-Watson model applies (see Figure 1 and Figure 2).

However, unlike in data warehousing, where data quality is an outcome of successful projects, one should see BI data quality as an(other) antecedent of successful BI projects because they rely on the quality of the data in the warehouse. Therefore, the modified BI organisational success model, based on the Wixom-Watson model, uses this construct as an independent factor.

FIGURE 1: Wixom-Watson model for data warehousing success.

FIGURE 2: Proposed business intelligence project success model.

A secondary objective is to test the extent to which these CSFs are generalisable. Therefore, the participants should rate the CSFs hypothesised in the theoretical research model relatively highly. How well the CSFs align between business and IT experts, between project-specific and general CSFs as well as with those CSFs highlighted in prior BI research literature has highlighted should also reflect this.

To provide a baseline for the second objective, the authors chose the Olbrich et al. (2012) study from Europe. It uses a comprehensive list of factors in three rounds of a Delphi study and covers three dimensions of interest: importance, variability and controllability. Therefore, the authors can convert the secondary research objective into the hypotheses below:

Hypothesis 1: the participants should rate the CSFs hypothesised in the theoretical research model highly.

Hypothesis 2: the relative importance of project-specific CSFs will align with the general BI CSF ratings.

Hypothesis 3: the relative importance of CSFs, as the South African participants rated them, will align with those of the European study. (Olbrich et al. 2012)

The authors held separate presentation sessions at each business unit to explain the purpose and objectives of the research and the benefit to the business units to the employees who participated. This was to ensure a high participation rate. The participants were from IT and business. Thirty-one participants received the initial email in the first round and 26 responded. This is a response rate of 84% and represented high quality stakeholders (Table 1). The authors regarded the sample size of 26 as sufficient to obtain results of material value (Yeoh & Koronios 2010).

The authors used the Olbrich et al. (2012) list of 25 factors for their initial list of CSF factors. They dropped five factors because they were not relevant for a single sector, single organisation study. They adapted five CSFs for this research. Because this research focused on business units, ‘corporate strategy’ became ‘business unit strategy’ and ‘influence of IT on corporate strategy’ became ‘influence of IT on business unit strategy’. They felt that the CSFs’ ‘sophistication of IT infrastructure’ and ‘sophistication of competitors’ BI technology’ were not clear. Therefore, they changed these CSFs to ‘technologically advanced IT infrastructure’ and ‘capability of competitors’ BI technology’ respectively. Because the description of ‘Data sources’ in the original study emphasised data quality, they renamed it ‘Data quality’ for this study.

The authors added four additional factors for the study. They regarded ‘business champion’ (a high-level champion from business) as separate from ‘top management support’ because they considered a business champion as much closer and involved with the project than the top managers were.

A ‘business case’, which defines the high-level scope and benefits, sets the framework of business expectations and the business sponsor, or champion, drives it for the project. ‘Data classification’ defines common meanings for concepts across the organisation or divisions. Because the BI system depends on data from the source systems, the degree to which data entities have the same meaning across the systems can affect the BI solution.

Adopting new BI initiatives vs. keeping the business going by doing upgrades in a maintenance (‘business as usual’) mode demonstrates the ‘organisational culture’ towards BI.

This resulted in a final list of 23 CSFs for the first round. In the second round, the authors added another CSF (‘user involvement’), because of participant input (see below).

A further unique improvement to the instrument is that, instead of merely providing a graded Likert scale (like from ‘very important’ to ‘not important’), the authors provided a detailed descriptive label for each of the five levels of importance. In addition, they asked participants to rate all CSFs twice: once for the respondents’ business unit’s specific BI project and once for the participant’s perception of BI projects in general.

In the second Delphi round, the authors also asked the respondents to rate perceived variability and controllability of the most important CSFs.

The motivation for using the Delphi method in those studies was that it:

has proven to be valuable in surfacing new issues and moving participants toward consensus. It is considered to be well suited to situations in which subjective and complex judgments are of interest, as opposed to precise quantitative results. (Olbrich et al. 2012:4151)

One can condense the perceptions of a disparate group into a collective view to reach consensus (Skulmoski, Hartman & Krahn 2007). However, Linstone & Turoff (2011) challenge this. They note that consensus can be an outcome, but is not essential. Untapped value also lies in analysing where the feedback diverges.

When one decides on the number or rounds of sampling, the target should be to achieve consistent and stable feedback. Therefore, researchers should be flexible and finish the sampling once they feel they have achieved this (Linstone & Turoff 2011). In the Olbrich et al. (2012) study (that targeted consensus), researchers felt that they had achieved moderate consensus after round 2, but continued with a third round of sampling. This resulted in the participation rate dropping from 27 to 13. They concluded that the participants might have been satisfied with their feedback from round two. With regard to the number of participants, two studies by Yeoh, Gao and Koronios (2007) and Yeoh & Koronios (2010) used a small sample of 10 to 15 participants, based on research by Adler and Ziglio (1996). They found that one could obtain satisfactory results from such small sample sizes.

The main driver for the project was to drive profitability and growth. Over a four-year period, the unit implemented the system in phases. The backend source systems range from mainframe to enterprise resource planning (ERP) systems. The unit used extract, transform and load (ETL) programs to populate the enterprise data warehouse with the backend system data. Some data transformation occurs at this stage.

The unit performs data processing using data cubes and other techniques to analyse data. It also uses predictive analytics software as part of the data analysis. A dashboard provides the data to the users using QlikView. This enabled users to drill down in the specific data they require. The plan was to roll out the system over time to approximately 200 users. Positive feedback has resulted in a very significant increase in requests to use the system. There are now over 670 active users on the system. The unit now presents a single version of the data. With the data now being easily accessible to the user, this system has also provided productivity improvements.

Business unit two: an operational key performance indicators reporting system. Business unit two implemented an operational BI system. This was in response to an operational efficiency drive in their back-office processing department. A consultant recommended a variety of key performance indicators (KPIs) for measurement.

The unit extended the back-office system to capture the additional information it required and established a data warehouse. It created an ETL program to extract the information for the data warehouse for online analytical processing (OLAP). The project took about one year because other projects had higher priorities for resources.

The outcome was a variety of reports that covered the workflow of the back-office agents. The unit delivered these reports in Microsoft Excel^® format. Some of the reports were complex and used cross-tabbed functionality. The unit regarded the system a success because it did not need to fill a number of vacancies. The solution has subsequently needed minimal maintenance.

Business unit three: a drill-down reporting system. Business unit 3 implemented a BI solution to increase the level of reporting. The unit previously reported using Excel ‘manually’.

Because of the volume of transactions, the unit was reporting only on transactions over a certain threshold and mainly at branches.

However, there were differences in opinion between the branches as to what they needed to report. The unit loaded data from five backend systems using an ETL process into their multidimensional database management system, Essbase. During this process, data transformation occurred.

The unit performed analyses of the data and initially produced a number of reports. From the reports, the unit identified a number of oversights in the manual reporting. It requested further reports. This significantly increased the coverage and granularity of the reporting.

Because the business unit is small, no business analyst was involved in the project. This resulted in issues between what the business expected and what IT thought they wanted. The project also had data quality issues that only emerged in production.

Notwithstanding these shortcomings, executive managers saw the value in the solution and have embarked on further BI projects.

The authors analysed the data and could generally match them with, or map them to, items already in the researcher-provided list (see Table 3).

In both Delphi rounds, participants rated each of the CSFs the authors provided using a detailed descriptive importance rating. Few participants changed their ratings in the second round. Therefore, one can assume that the participants reached a sufficient degree of consensus after only two rounds.

Figure 3 gives the ratings of the second round for the general CSFs (not project specific). One can use several criteria to rank the CSFs given their importance scores. Figure 3 ranks CSFs using the ‘Very Important’ scores as the primary sorting key and, within that, ‘Decidedly Important’ as a secondary criterion. This order aligns quite well (r = .965) with a weighted average score based on the traditional ‘equidistant’ Likert weights except that, in the latter case, business unit strategy and technical capability rank markedly higher.

FIGURE 3: Importance ratings for each Critical Success Factor (round 2) by average and detailed breakdown.

These factors and their relative importance align well with the European study (see Hypothesis 3 below). Most surprisingly, the empirical findings align almost perfectly with the theoretical framework.

One should note that, in this survey, the authors split ‘resources’ into two composite factors: financial resources versus technical resources and capabilities. If one takes a high ranking of either of the two resource types as valid, then it is extremely satisfying to find that the respondents ranked the five key implementation factors in the theoretical framework within the top seven general BI CSFs.

This finding is statistically highly significant. One calculates the p-value using the probability of finding five specific values out of 23 in a set of (in this case, the top) seven factors. This follows a hypergeometric distribution HyperGeomDist [the number of required values (‘successes’) in sample = 5, sample size = 7, number of allowed values (‘successes’) in population 5, population size = 23] = .000624. Because one allows for either of two different possible resource types, but excludes the double counting of the cases where one includes both resource types in the top 7, p = 2×.000624–.000069=.001179 (that is, statistically highly significant). Therefore, one must support the hypothesis that the findings conform strongly to the theoretical framework.

The business units were at very different stages in their BI maturity. This made the study more challenging.

Business unit one had completed many iterations of a BI program over a period of four years. There had been significant success, which the increase in user demand and pervasiveness within the business unit measured. Business unit two had completed their first project using a data warehouse. The backend data source was the operational system for their back office. Even though the managers saw the implementation as a success, they always regarded it as a once-off project and undertook no subsequent major BI initiatives. The executive sponsor of business unit three regarded the unit to be at a very early stage of experience with their implementation of BI solutions. Most likely the consumers of this research will also be at different stages in their BI maturity. Therefore, one can regard the cross section of selected business units as representative of the wider market.

The challenge, with such a diversity of BI implementation experiences, is to be able to draw generalisable conclusions. The first test was to compare the project-specific ratings of each business unit with the overall general rating of the three business units. The authors chose this comparison to determine how the rating specific to each project would correlate with ratings based on all the participants’ general experience of BI. The authors made the comparison using Spearman’s ranking correlation and Pearson’s correlation. The project specific ratings for the three business units correlated with the mean overall general ratings for both rounds.

As the authors expected, business unit one obtained the highest correlation. The other two business units also had strong correlations for both rounds. Business unit three, as the authors expected, had the lowest correlation. Therefore, this supports Hypothesis 1. A cautionary note is that, because business units one and two made up 81% of the sample, it can introduce bias and result in an inflated correlation.

The surprisingly high correlations prompted the question at the interviews of whether the project specific ratings had materially influenced the general ratings. Two of the interviewees representing business units two and three believed there was bias for their business unit’s feedback when completing the survey. According to interviewee 3: ‘the project is almost all that you know, I think our people would definitely lean back to the project’. Interviewee 4 said: ‘I think there would be in any case because each project has its own specific requirements; the general would be an average across many projects’.

Based on the feedback from interviewee 3 and interview 4, one can deduce that, for the business participants, there was to some degree a bias in the general ratings based on their experience of their specific project used in this research.

In the literature the authors reviewed, they posited that the CSFs applied generally across industries and types of projects. The Olbrich et al. (2012) study made no distinction between the types of BI projects, attempted to include a wide as possible spectrum of industries, and limited the number of participants per industry. Other researchers (Presthus et al. 2012; Yeoh & Koronios 2010) identified similar factors that contributed to BI implementation success in different industries.

The authors had mixed findings. The only correlation they found was between business units one and two for both the project and general ratings. There were no significant correlations with business unit three. One could attribute this to the unit’s lack of experience in BI.

The authors undertook further investigation into these finding at the interviews. Interviewee 1 and interviewee 2 believed that there should be a standard generic list applicable to any type of BI project, irrespective of its industry. According to one of the interviewees:

‘I think in BI there are certain things that you can use in a generic way or generic approach and I think key success factors are definitely one of those’. (Interviewee 1)

Interviewee 1 also mentioned that, in addition to the generic factors, one could add others:

‘I do think that yes your generic list could probably cover about 80%’ and ‘Yes they will start listing their own key success factors from experience gained so there’s a difference between the theory and the actual experience’. (Interviewee 1)

When the authors asked interviewee 2 whether they would use a subset of the factors from the study for the next project, the response was:

‘No, I wouldn’t even take a subset of this, I mean most of these things are actually absolutely spot on, a lot of the stuff is very important’. (Interviewee 2)

These views contradicted those of interviewee 3 and interviewee 4. Their view was that there would be project influences on which factors would be important. Interviewee 3 noted: ‘for me the critical success factors of a BI project would differ between a small and a large’ business unit. When the authors asked interviewee 4 to comment on the differences in the top ten factors highlighted between the business units, interviewee 4 responded that: ‘I would say their results indicate why I expect a difference between the organisations depending on their specific needs’.

The authors’ findings correlated with a study on CSFs by Olszak and Ziemba (2012). It dealt with Small and Medium-sized Enterprises (SMEs) in Upper Silesia, Poland. They found that, whilst most of the CSFs they identified aligned with those of other researchers, there were notable differences like budget, available skills and well-defined user expectation.

The authors excluded business unit three from this analysis because there were only five participants: two from IT and three from business.

For business unit one, the IT ratings saw the categories of ‘committed management support and champion’ (‘business unit strategy’, ‘influence of IT on business unit strategy’, ‘top management support’ and ‘business champion’) and ‘business vision’ (‘business case’ and ‘BI strategy’) make up the bulk of the top ten. The business participants’ top ten was also heavily weighted towards the same two categories. Data quality was also in their top ten. There were six CSFs common to both sets of top ten ratings. An interesting view was that business saw the technical capability of IT to deliver the BI solution as their most important CSF.

The business participants from business unit one rated ‘competitor’s BI capability’ in their top ten factors. Interviewee 1 noted that, as IT, ‘we don’t do much work around what the competitors are doing, these guys (business) need to tell us that’ and:

‘we don’t see what the brokers are getting, so to me the competitors are not that important. What is important is that I understand how we can be better where business is comparing us to the competitors’. (Interviewee 1)

This is an example of where business and IT participants view some of the CSFs from a different perspective, resulting in different levels of importance.

For business unit two, there were seven CSFs common to both sets of top ten ratings. The average ratings between the two datasets were closer than for business unit one. They again placed clear importance on the ‘committed management support and champion’ category. Information technology and business both rated data quality as the top CSF. As interviewee 4 noted: ‘data is a very high quality, very high importance to the customer, damage to the reputation would be very high’.

Business unit two also rated management methods high because the main aim of the BI system was to produce reports that show whether the business was meeting the efficiency metrics it specified. As interviewee 4 noted: ‘The BI project was very much around the balance score card and the performance of the environment, so for this project those were rated really high’.

Consistent with the efficiency drive, it also rated process automation in the top set for both sets of participants.

Information technology also rated the literacy the users required as important. Moreover, according to one of the interviewees:

‘The process automation and literacy, our users need to have a good understanding of the business to get through the requirements and our people also need a good understanding of the business in order to deliver value’. (Interviewee 4)

For business, articulating the vision was important with ‘business case’ and ‘BI strategy’ featuring in their top ten.

There is a clear similarity in the categories of CSFs that are important to both IT and business in both business units. They saw the roles the top managers played in supporting the project as influential in achieving the business goals for the system. Therefore, for a BI project’s outcome to succeed, one should view it as a business project that extracts value from business data. The high importance both business and IT participants placed on CSFs in the ‘committed management support and champion’ and ‘business vision’ categories indicates the significant role non-technical CSFs play in the success of a BI implementation. This is consistent with the findings of Adamala and Cidrin (2011). Another study, by Yeoh et al. (2007) in the field of engineering asset management, also noted that, in industry, the top CSFs were weighted towards business skills and manager involvement.

Olbrich et al. 2012 based their first comparison on ratings and the second comparison on rankings. Because the authors of this article based all the South African results on ratings, they calculated the rankings for the second round using the mean ratings. They based their comparisons on the sixteen CSFs that were common to both studies for round one and the nineteen CSFs that were common to both studies for round two.

The statistical tests showed a good correlation between the two studies with a rank-correlation coefficient of .577 (statistically significant at p = .01).

However, whilst the South African study comprised IT and business participants, the European study comprised only IT participants who were senior managers and project managers. Therefore, the authors of this article refined the analysis to distinguish whether there is a correlations between the European and the rankings of the South African IT (H_3a) and business participants (H_3b) respectively. Not surprisingly, the ratings of the IT participants showed a much stronger correlation (.579), with the p-value of .0099 being highly significant. The ratings of the business participants, on the other hand, showed only a weak, statistically non-significant correlation (.354, p = .118) between the datasets. This illustrates clearly how the criteria researchers use to select samples can influence research outcomes.

FIGURE 4: Perceived variability of top critical success factors.

FIGURE 5: Perceived controllability of top critical success factors.

Only seven factors for both dimensions occurred in both studies. As the European study used ranking for rounds two and three and this study used ratings, the authors of this article drew the comparison with round one of the European study. For neither of the two dimensions could they establish a statistically significant correlation. The respective correlation coefficients were .3055 for variability and .1868 for controllability.

When the authors examined the absolute values, they seemed to show that the South African participants perceive that there is less variability in their environments and that they have more control over the factors. This could be because there is bias in the sample because ten of the participants in round one come from business unit one, whose BI implementation has been very successful. Because the researchers conducted the European study in different industries, the authors believe that the European ratings for the variability and controllability dimensions may be a truer reflection of the reality organisations generally experience.

The literature review provided a number of studies that researchers had conducted in different parts of the world (Olbrich et al. 2012; Yeoh et al. 2007). These studies provided insights into the CSFs that were key to enabling successful BI project implementation. A review of South African BI research did not reveal any research dedicated to CSFs in the BI domain. This presented an opportunity to add to the body of existing knowledge and provide findings that are relevant to the South African context.

The theoretical framework the authors used for the study followed the model of Wixom and Watson (2001) for their business warehouse project success. They modified it for the context of business intelligence as well as to focus on organisational and project implementation success instead of technical success (Figure 2).

The authors used a mixed-method approach of a survey and interviews. The survey followed the Delphi method approach with the participants primarily required to rate the CSFs for importance based on a Likert scale for their business unit’s project and separately for the context of BI in general. The participants rated a subset of factors for variability and controllability. The interviews were semi-structured and asked open-ended questions. The authors asked follow-up questions when interesting points surfaced.

The study identified that the most important CSFs for the participating business units belonged to the categories of ‘committed management support and champion’, ‘business vision’, ‘user involvement’ and ‘data quality’. The highest rated CSFs for the importance dimension for the BI in the general context were ‘data quality’, ‘business case’ and ‘influence of IT on business unit strategy’, whilst the highest rated in the project context were ‘data quality’, ‘top management support’ and ‘business champion’.

These findings were almost a perfect validation of the theoretical framework, because all five factors the authors postulated for organisational and project implementation success feature amongst the seven most highly ranked CSFs (p = .00125).

The authors achieved a moderate correlation between the overall rating of this study for the BI in the general context with the European study. There were minor but significant differences in the ratings the IT and the business experts gave. Participants also identified a few differences between project-specific and ‘generic’ BI CSFs.

The participating business units had achieved different levels of BI maturity. Project-to-project comparisons reflected this, where the ratings resulted in correlations between only two of the business units. This difference in the BI maturity of the business units is a natural expectation and adds value to the research for consumers. If the authors had selected only organisations with very mature BI approaches, it may have limited the relevance for those starting with BI or those still developing their expertise.

Because one can assume that there will be varying levels of BI maturity in other financial organisations in South Africa, the authors expect that one can generalise their findings to other South African financial institutions. However, further research would need to be undertaken before one can generalise these results to other industries.

From a theoretical perspective, the authors’ proposed theoretical framework fared very well. All five factors featured in the top seven CSFs the respondents identified. This means that the framework can provide a ‘short-cut’ summary of the key CSFs and validates the generalisability of the findings. However, future models may look at incorporating the other two key variables the authors uncovered: business case and IT influence on business unit strategy.

What did emerge from the interviews was that none of the projects had explicitly identified upfront the CSFs together with the relevant metrics they needed for their projects to succeed. However, the respondents did address some implicitly. Encouragingly, some interviewees noted that, now that they had had exposure to the list of CSFs, they would use it as a reference for future projects. Additional research could compare the outcomes of BI projects of organisations that do explicitly identify the key CSFs together with metrics upfront for their projects versus those that do not.

Adler, M. & Ziglio, E., 1996, Gazing into the oracle: the Delphi method and its application to social policy and public health, Jessica Kingsley Publishers, London.

Bijker, M., 2010, ‘Understanding the Factors that Influence Pervasiveness of Business Intelligence in South Africa’, Technical report, Dept. of Information Systems, University of Cape Town, Cape Town.

Blenkhorn, D.L. & Fleisher, C.S., 2007, ‘Performance Assessment in Competitive Intelligence: An Exploration, Synthesis, and Research Agenda’, Journal of Competitive Intelligence and Management 4(2), 4–21.

DeLone, W.H. & McLean E.R., 1992, ‘Information systems success: the quest for the dependent variable’, Information Systems Research 3(1), 60–95. http://dx.doi.org/10.1287/isre.3.1.60

Gartner Research — BI Revenue, 2012, ‘Gartner Says Worldwide Business Intelligence, Analytics and Performance Management Software Market Surpassed the $12 Billion Mark in 2011’, viewed 5 December 2012, from http://www.gartner.com/it/page.jsp?id=1971516

Hartley, K. & Seymour, L.F., 2011, ‘Towards a framework for the adoption of business intelligence in public sector organisations: the case of South Africa’, SAICSIT Annual Research Conference, Cape Town, South Africa, October 3–5, 2011, pp. 116–122.

Hawking, P. & Sellitto, C., 2010, ‘Business Intelligence (BI) Critical Success Factors’, 21st Australian Conference on Information Systems, Brisbane, Australia, December 1–3, 2010, AIS Electronic Library.

Linstone, H.A. & Turoff, M., 2011, ‘Delphi: A brief look backward and forward’, Technological Forecasting & Social Change 78(9), 1712–1719. http://dx.doi.org/10.1016/j.techfore.2010.09.011

Lönnqvist, A. & Pirttimäki, V., 2006, ‘The Measurement of Business Intelligence’ Information Systems Management 23(1), 32–40. http://dx.doi.org/10.1201/1078.10580530/45769.23.1.20061201/91770.4

Luftman, J. & Ben-Zvi, T., 2010, ‘Key Issues for IT Executives 2010: Judicious IT Investments Continue Post-Recession’, MIS Quarterly Executive 9(4), 263–273.

Lutu, P.E. & Meyer, B., 2008, ‘The successful adoption and usage of business intelligence in public sector organisations: an exploratory study in South Africa’, Proceeedings of IFIP WG 9.4-University of Pretoria Joint Workshop, Pretoria, South Africa, September 23–24, 2008, pp. 164–173.

O’Brien, J. & Kok, J.A., 2006, ‘Business Intelligence and the telecommunications industry: can business intelligence lead to higher profits?’, South African Journal of Information Management 8(3), 1–16.

Olbrich, S., Pöppelbuß, J. & Niehaves, B., 2012, ‘Critical Contextual Success Factors for Business Intelligence: A Delphi Study on Their Relevance, Variability, and Controllability’, 45th Hawaii International Conference on System Sciences, Hawaii, USA, January 4–7, 2012, pp. 4148–4157.

Olszak, C.M. & Ziemba, E., 2012, ‘Critical Success Factors for Implementing Business Intelligence Systems in Small and Medium Enterprises on the Example of Upper Silesia, Poland’, Interdisciplinary Journal of Information, Knowledge, and Management 7, 129–150.

Pellissier, R. & Kruger, J-P., 2011, ‘Understanding the use of strategic intelligence as a strategic management tool in the long-term insurance industry in South Africa’, SA Journal of Information Management 13(1), Art. #426, 13 pages. http://dx.doi.org/10.4102/sajim.v13i1.426

Pirttimäki, V.H., 2007, ‘Conceptual analysis of business intelligence’, SA Journal of Information Management 9(2), 17 pages.

Ponelis, S.R., 2011, ‘An exploratory study of business intelligence in knowledge-based growth small, medium and micro-enterprises in South Africa’, PhD thesis, Dept. of Information Technology, University of Pretoria, South Africa.

Presthus, W., Ghinea, G. & Utvik, K.-R., 2012, ‘The More, the Merrier? The Interaction of Critical Success Factors in Business Intelligence Implementations’, International Journal of Business Intelligence Research 3(2), 34–48. http://dx.doi.org/10.4018/jbir.2012040103

Ranjan, J., 2008, ‘Business justification with business intelligence’, The Journal of Information and Knowledge Management Systems 38(4), 461–475.

Skulmoski, G.J., Hartman, F.T. & Krahn, J., 2007, ‘The Delphi Method for Graduate Research’, Journal of Information Technology Education 6, 1–21.

Smith, D. & Crossland, M., 2008, ‘Realizing the value of Business Intelligence’, Advances in Information Systems Research, Education and Practice, IFIP Vol. 274, 163–174.

Wixom, B.H. & Watson, J., 2001, ‘An Empirical Investigation of the Factors Affecting Data Warehousing Success’, MIS Quarterly 25(1), 1–24. http://dx.doi.org/10.2307/3250957

Yeoh, W. & Koronios, A., 2010, ‘Critical Success Factors for Business Intelligence Systems’, Journal of Computer Information Systems 50(3), 23–32.

Yeoh, W., Gao, J. & Koronios, A., 2007, ‘Towards a Critical Success Factor Framework for Implementing Business Intelligence Systems — A Delphi Study in Engineering Asset Management Organizations’, IFIP TC 8 WG 8.9 International Conference on Research and Practical Issues of Enterprise Information Systems, Beijing, China, October 14–16, 2007, pp. 1353–1367.