Background: The continued in failure of agile and traditional software development projects have led to the consideration, attention and dispute to critical success factors that are the aspects which are most vital to make a software engineering methodology fruitful. Although there is an increasing variety of critical success factors and methodologies, the conceptual frameworks which have causal relationship are limited.

Objective: The objective of this study was to identify and provide insights into the critical success factors that influence the success of software development projects using agile methodologies in South Africa.

Method: Quantitative method of collecting data was used. Data were collected in South Africa through a Web-based survey using structured questionnaires.

Results: These results show that organisational factors have a great influence on performance expectancy characteristics.

Conclusion: The results of this study discovered a comprehensive model that could provide guidelines to the agile community and to the agile professionals.

Introduction

It has long been understood that a large portion of software development projects fail. Recent estimates note that approximately 65% of software development projects, in general, do not meet their time frame and incur high costs to the organisation (Bossini & Fernández 2013). Organisations are spending a large amount of money adopting agile development processes with an expectation that the software projects are going to be completed on time with less cost (Ambler 2009; Chow & Cao 2008). However, many software development projects, even those adopting an agile methodology, continue to incur costs to the company which were not budgeted for, and are completed beyond the initially agreed time frame (Dyba & Dingsøyr 2015).

The selection of inappropriate methodologies to manage software development projects contributes to the failure of these projects (Bossini & Fernández 2013; Stankovic et al. 2013). Currently, there is confusion over which software development process to choose in different circumstances. Consequently, there is a need for software development managers to understand when it is appropriate to use agile and when to use traditional methodologies (Taromirad & Ramsin 2008).

Although a number of software development methodologies exists, each with related critical success factors (CSFs), software project managers find it difficult to select the most appropriate one (Nguyen 2016). This is partially because software development professionals tend to be passionately devoted to and subjective towards the software engineering methodologies in which they have vast experience (Bossini & Fernández 2013).

When the CSFs of agile software development projects are not logically connected with project objectives and their specific environment, this leads to failure to deliver software projects on time, and at a higher cost to the software project than budgeted for by the organisations (Chow & Cao 2008; Misra, Kumar & Kumar 2009). There is no complete research framework to identify and develop insights into all relevant CSFs for agile development and their linked constructs which have a vital meaning to the software development community and organisations (Dyba & Dingsøyr 2015; Misra et al. 2009).

Problem statement

Software development projects adopting agile methodologies face a number of challenges which have emerged through various studies, including:

Primary research question

Secondary research questions

Background

Despite the potential benefits of using agile, software development professionals have been slow in adopting agile methodologies. In a global study, Chan and Thong (2009) discovered that 60% of companies surveyed were using neither agile nor any traditional methodologies, only 6% followed a methodology rigorously and 79% of those not using any methodology did not intend to adopt one. One of the reasons for the lack of acceptance of agile specifically is that early adopters of technology are highly resistant to changes to new technology. Further, agile methodologies are assumed to be universally applicable and people assume that they can be adjusted to software development during a project. The acceptance of agile software development methodologies remains a persistent challenge that attracts agile professionals’ attention (Chow & Cao 2008).

Joseph (2013) explained that more than 65% of software projects in South Africa were perceived as being failures and facing challenges in 2013 (as shown in Figure 1). This might have been because software development projects were not completed in time or within the estimated budget. Joseph (2013) further explained that 34% of the software development projects were perceived as successful (as shown in Figure 1). In a similar study, Marnewick (2012) found that 59% of the software development projects surveyed in 2011 were perceived to be successful and 41% of the software projects were perceived to be unsuccessful and facing challenges in South Africa (as shown in Figure 1).

The Industrial Development Corporation (2012) predicts an increase in software project spending in South Africa of R154 billion in 2016. This implies that R18.48 billion could be wasted in 2016 based on the 2011 failure rate shown by Marnewick (2012). This amount could potentially increase to a staggering R49.28 billion if the current 32% failure rate is considered (as shown in Figure 1) (Joseph 2013). Joseph (2013) showed that approximately R30.8 billion could be lost in failed software projects (as shown in Figure 1). This raises this concern whether all software development project managers are professional who know what they are doing.

All software development project success depends on several factors such as having a software project leader with an effective management style and decision making. The main building blocks for the success of an agile project specifically are time, acceptance and understanding of agile methodologies, quality, budget and scope. On the other hand, there are several factors which agile management and agile professionals lack even though they have experience from previous agile software projects, namely people factors, organisational factors, culture factors, process factors, project factors, political factors and technical failure factors (Chow & Cao 2008; Cohn & Ford 2003; Dyba & Dingsøyr 2009; Koch 2005; Lindvall et al. 2004; Misra et al. 2009).

A quantitative research method was used in this study to understand the practical experience of agile professionals with the agile project success exploring the human, project, process, organisational, technical, political and technological factors in South African organisations (Table 1). Government in South Africa and agile organisation has created a conducive environmental for creation and growth or transformation of enterprises. This consists of good communication, good accessibility of resources, project preparation, budget allocation and modification control process of software projects, just to name a few (Chow & Cao 2008; Dyba & Dingsøyr 2009; Joseph 2013; Misra et al. 2009). However, there is limited information available pertaining to the CSFs that have an effect on agile software development projects (Joseph 2013).

The scope of the current study is solely within South Africa. This study has implications for positive social change in organisations as they will be better able to understand the CSFs of agile software development projects. This study will also enable organisations to develop strategies to improve agile software development projects and cost benefits leading to higher profitability and productivity of software projects.

Methodology

The study used both quantitative methods of collecting data. An administering online questionnaire was distributed to all the 800 agile professionals and agile management in South African organisations. Four-hundred and sixty questionnaires copies were returned. This gave a respond rate of 58%. Quantitative data were analysed statistically using AMOS version 23.0 and SPSS version 23.0.

The sampling technique used in this research study is the purposive sampling method. The purposive sampling technique is a method whereby an experienced individual selects the sample based on his or her judgement about some appropriate characteristics required of the sample members (Creswell 2013).

Formulation of hypothesis

To guide the research and data analysis for the categories, the following hypotheses were formulated:

Performance expectancy

The factors that affect agile software development projects include perceived usefulness. Perceived usefulness has been discovered to affect user perceptions of different technology innovations, such as mobile commerce and agile software development projects (Chan & Thong 2009; Venkatesh et al. 2003).

Perceived usefulness (performance expectancy) refers to the degree to which an individual expects that adopting software development methodologies will improve the individual’s job performance (Hardgrave, Davis & Riemenschneider 2003). Prior studies on acceptance of software development methodologies (Hardgrave et al. 2003) have found that perceived usefulness is a significant factor in predicting the usage and success of software development methodologies. Generally, prior research suggested that the more a methodology is perceived as enabling an increase in job performance, the more likely that it will be accepted. This leads to the following hypothesis:

Organisation factors

Numerous researchers such as DeLone and McLean (2002) and Curtis and Payne (2008) suggest that organisational factors play an important role in the use of agile methodologies which lead to positive performance expectancy and success of agile software development projects. Similarly, research on agile methodologies showed that one of the top most CSFs of agile methodologies is organisational factors (DeLone & McLean 2002; Koch 2005). Therefore, scientists have found that top executive or management support as one of the first priority with regard to the agile methodologies CSFs (Chow & Cao 2008). These scientists claim that executive or top management needs to show and tangibly give sustenance in implementation of the agile methodologies in the project (Ramesh, Cao & Baskerville 2010). This sustenance can be in the form of giving help or support to users such as team members of the group, hire the services of consultants, train users, psychological and financial and moral support. This leads to the following hypotheses:

People factors

According to Triandis (1980) noted that, people beliefs play a significant part for users to believe a system as easy or difficult to use. This explains that when people believe that a system is easy to implement, they will implement it and if not they will avoid it (DeLone & McLean 2002). Therefore, the easier users see a system as easy to use and they go ahead to use it. This means that the easier it will be for them to understand its paybacks. Likewise, users’ viewpoints have an effect on their societal beliefs. This leads to the following hypothesis:

Technological, process and project factors

In this research, process factors, technological factors and project factors can have direct influence on success of the agile software development project, respectively (Augustine et al. 2005; Ceschi et al. 2005; Marnewick & Labuschagne 2009; Misra et al. 2009). This leads to the following hypotheses:

Data presentation and analysis

Reliability, validity of the constructs and correlation

This study used discriminant and convergent validity to determine, the degree of correlation between the decision variables and other measures that have been predicted in theory to correlate with them (Hartung & Knapp 2005). Furthermore, to determine whether these decision variables are not correlating with those other variables that have been theorised not to correlate with them (Pallant 2013). Decision variables’ reliability was also measured using Cronbach’s alpha.

Reliability of the constructs

Before testing for the discriminant convergent validity and correlation, the decision variables were evaluated for the reliability. Cronbach’s alpha (α) was used to check the internal consistency of the decision variables (Pallant 2013). The remaining values of the decision variables which are retained were above 0.7, meaning that their corrected item-total correlation is significant as shown in Table 2 (Pallant 2001 as cited by Chiyangwa & Trish 2016). The findings indicate that the Cronbach’s alpha values of all items range from 0.819 to 0.934, with an overall internal consistency reliability of 0.920 for 33 items.

Convergent and discriminant validity and correlation

Convergent validity defines the proportion of variance for each factor that are related (Al Tamimi 2014; Hair et al. 1995; Pomykalski, Dion & Brock 2008). Convergent validity is determined from the computation of the composite or construct reliability (CR) and variance extracted (VE) for each indicator of the construct. As recommended by Hartung and Knapp (2005) and Dion (2008), composite reliability and the average VE (AVE) were used to assess the convergent validities. While the discriminant validity was considered by examining whether or not the squared root of AVE exceeds the correlations between constructs and the reliability was evaluated by examining the internal consistency reliability as recommended by Venkatesh et al. (2003).

The SPSS version 23.0 and AMOS version 23.0 were computed and the generated results were summarised in Table 3; all factors in the measurement model had acceptable composite reliability and convergent validity because all factor loadings (the indicators’ standardised loadings [λ]) are significant (p < 0.001), the composite or construct reliabilities exceeding acceptable criteria of 0.7, and the AVEs were greater than the threshold value of 0.5 in all cases (Fabrigar et al. 2010; French & Finch 2006). Table 3 is constructed where diagonal elements are the square roots of AVE, and off-diagonal elements are correlations between decision variables (constructs). The discriminant validity and reliability were supported because Table 3 shows that all diagonal elements were higher than the off-diagonal elements in the corresponding columns and rows as well as internal consistency reliability were above 0.7.

Structural equation results

The critical ratio values of the path between the hypothesised decision variables express that their paths are significant except that of hypothesis H2b, H3b, H5a, H5b and H6b. This means that the proposed hypothesis Table 4 also shows hypothesised relationship among H6a (β = -0.041; p < 0.05; project factors influence performance expectancy factors), H4a (β = 0.036; p < 0.05; process factors influence performance expectancy factors), H3a (people factors influence performance expectancy factors) and H2a (β = 0.721; p < 0.05; organisational factors influence performance expectancy factors) for performance expectancy is significant and supported. Lastly, hypothesis H4b (β = 0.230; p < 0.05) between process factors and success factors are supported meaning that it is significant as shown in Table 4.

The predictions were measured by using the indirect and direct effects of the decision variables as theorised in the model. The extracted standardised coefficient weights output from AMOS version 23.0 is displayed in Table 4. The standardised coefficient weights represent the direct effects between decision variables. Relying on direct effects, AMOS version 23.0 was computed to produce a matrix table from which the indirect effects of the decision variables may be explained and extracted.

Organisational factors have the highest effect towards performance expectancy (H2a; β = 0.721; p < 0.05). The framework decision variables explain 18.0% of variance with regard to success and 8.0% of variance with regard to performance expectancy factors of agile software development project. The total effect of each decision variable on success factors can be observed as the effect that decision variable has towards success factors without or with a mediating variable being involved. The direct effect is the effect a decision variable has on a dependent decision variable in this case success factors with no mediating variable. As from the model in Figure 2, performance expectancy and process factors where hypothesised to have direct effects to the success of agile software projects as shown in Figure 2.

Discussion of results

Performance expectancy

Project factors have a positive effect on performance expectancy factors for the agile software development project success which was significant (H6a). The results exposed that agile user’s performance expectancy to practice agile methods is highly dependent on the dynamic, accelerated schedule, project type that has variable scope with emerging requirements and project nature that has a non-life-critical software project; although it could be business mission–critical software, project had no multiple, independent teams working together, project had up front, detailed cost evaluation completed and approved and project had a small team size.

Process factors have a positive effect on performance expectancy factors for the agile software development project success which was significant (H4a). The outcomes exposed that agile user’s performance expectancy to practice agile methods is highly dependent on the agile project management style, agile-oriented requirement process, project scope and objectives that are well -defined, projects which are honoured through regular working schedule, an agile-friendly progress tracking mechanism, and use flexible time-boxing or rapid-pace progress measurement techniques instead of document milestones or work breakdown structure, strong communication focus and rigorous communication schedule, agile-oriented configuration management process, strong customer commitment and presence and customer representative on the project to have a full authority and knowledge to make decisions on-site.

People factors have a positive effect on performance expectancy factors for the agile software development project success which was significant (H3a). The results showed that agile user’s performance expectancy to practice agile methods is highly dependent on the project that imposed a well-defined coding standard upfront, the project that pursued simple design, the project that pursued vigorous refactoring activities to ensure that the results are optimal and to accommodate well all changes in requirements, the project that maintained the right amount of documentation for agile purpose, the project that followed continuous and rigorous unit and integration testing strategy for each and every iteration, the project that delivered working software regularly within short periods of time, the project that delivered most important features first, the project that employed proper platforms, technologies and tools suitable for agility practice and the project that provided appropriate technical or technological training to the team, including training on subject matter and agile processes.

Organisational factors have a positive effect on the performance expectancy factors for the agile software development project success which was significant (H2a). The results exposed that agile user’s performance expectancy to practice agile methods is highly dependent on the project team working in a facility with proper agile-style work environment; all team members worked in the same location for ease of communication casual, constant and contact; organisation that had a cooperative culture instead of a hierarchal culture; organisation that had an oral culture placing high value on fluid, face-to-face communication style; and organisation that had a reward system that was appropriate for agile behaviour. Agile methodology was universally accepted in the organisation, and the project had a committed sponsor or a committed organisation manager and received strong management or executive support. In the urban areas of eight provinces in South Africa, several measures and campaigns have been put in place to promote agile software projects.

Success of agile software projects

Performance expectancy factors have a positive effect on the success of agile software development projects (H1); this hypothesis was supported. The findings are in compliance with Venkatesh et al. (2003) which supported that agile approach enables to accomplish tasks more efficient, useful and productive and increase chances of finding promotion in order to deliver good product or project software projects results, meet all requirements and objectives, deliver software projects on time and deliver software projects within estimated effort and cost of software projects.

Process factors have a significant positive effect on the success of agile software development projects (H4b), this hypotheses was supported. Venkatesh et al. (2003) supported that the validity of this hypothesis has been examined and confirmed true by Tibenderana et al. (2010), which supported that agile approach enables to accomplish tasks as follows:

Conclusion

The research questions were reliable and valid as measured by composite reliability, Cronbach’s alpha, exploratory factor analysis and confirmatory factor analysis. The research study examined the adoption of agile management and expert regarding success factors through the use of the study model. It was found that performance expectancy factors and process factors have a causal relationship with success factors. Furthermore, organisational, process, people and project have a direct effect on the performance expectancy factors of practicing agile software development projects. These results show that the dominant factors that determine whether individuals practice agile software development projects are organisational factors towards performance expectancy factors. Organisational factors regarding agile software development project were found to be statistically significant but the performance expectancy is the single most important factor which needs to be taken into consideration when promoting software projects in South Africa.

These findings will assist agile management and experts and researchers to concentrate on the most significant activities so that the most CSFs can be better understood and applied in the organisation. With regard to the structural equation modelling, organisational factors were found to be the dominant factor in the accomplishment of performance expectancy factors.

Acknowledgements

Tawanda B. Chiyangwa would like to thank the University of South Africa and Ernerst Mnkandla for his supervision and guidance.

The authors declare that they have no financial or personal relationship(s) that may have inappropriately influenced them in writing this article.

E.M. was responsible for supervision and guidance and T.B.C. carried out the whole research.

References

Al Tamimi, A., 2014, ‘Empirical investigation on success factors in adapting agile methodology in software development at public organisations’, M.Sc. dissertation, The British University in Dubai, Dubai. https://bspace.buid.ac.ae/bitstream/1234/657/1/120152.pdf

Ambler, S., 2009, ‘The agile scaling model (ASM): Adapting agile methods for complex environments’, Environments, (December), 1–35.

Augustine, S., Payne, B., Sencindiver, F. & Woodcock, S., 2005, ‘Agile project management: Steering from the edges’, Communications of the ACM 48(12), 85–89.

Bossini, J. & Fernández, A.R., 2013, ‘Using agile methodologies in people management’, RPM 10(1), 33–42.

Ceschi, M., Silitti, A., Succi, G. & De Panfilis, S., 2005, ‘Project management in plan-based and agile companies’, IEEE Software 22(3), 21–27.

Chan, F.K. & Thong, J.Y., 2009, ‘Acceptance of agile methodologies: A critical review and conceptual framework’, Decision Support Systems 46(4), 803–814. https://doi.org/10.1016/j.dss.2008.11.009

Chiyangwa, T.B. & Alexander, P.T., 2016, ‘Rapidly co-evolving technology adoption and diffusion models’, Telematics and Informatics 33(1), 56–76. https://doi.org/10.1016/j.tele.2015.05.004

Chow, T. & Cao, D.B., 2008, ‘A survey study of critical success factors in agile software projects’, Journal of Systems and Software 81(6), 961–971. https://doi.org/10.1016/j.jss.2007.08.020

Cohn, M. & Ford, D., 2003, ‘Introducing an agile process to an organisation’, Computer 36(6), 74–78.

Creswell, J.W., 2013, Research design: Qualitative, quantitative, and mixed methods approaches, Sage, London.

Curtis, M.B. & Payne, E.A., 2008, ‘An examination of contextual factors and individual characteristics affecting technology implementation decisions in auditing’, International Journal of Accounting Information Systems 9(2), 104–121.

DeLone, W.H. & McLean, E.R., 2002, ‘Information systems success revisited’, in Proceedings of the 35th Annual Hawaii International Conference on System sciences, 2002. HICSS, pp. 2966–2976, IEEE, Big Island, HI, January 07–10, 2002.

Dion, P.A., 2008, ‘Interpreting structural equation modeling results: A reply to Martin and Cullen’, Journal of Business Ethics 83(3), 365–368. https://doi.org/10.1007/s10551-007-9634-7

Dyba, T. & Dingsoyr, T., 2009, ‘What do we know about agile software development?’, IEEE Software 26(5), 6–9. https://doi.org/10.1109/MS.2009.145

Dyba, T. & Dingsøyr, T., 2015, Agile project management: From self-managing teams to large-scale development, paper presented at the 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Florence, Italy, May 16–24, 2015.

Fabrigar, L.R., Porter, R.D. & Norris, M.E., 2010, ‘Some things you should know about structural equation modeling but never thought to ask’, Journal of Consumer Psychology 20(2), 221–225.

French, B.F. & Finch, W.H., 2006, ‘Confirmatory factor analytic procedures for the determination of measurement invariance confirmatory’, Structural Equation Modeling 13(3), 378–402.

Hair, J.F., Black, B., Anderson, R. & Tatham, R., 1995, Multivariate data analysis: Text and readings, Prentice-Hall, Inc., Upper Saddle River, NJ.

Hardgrave, B.C., Davis, F.D. & Riemenschneider, C.K., 2003, ‘Investigating determinants of software developers’ intentions to follow methodologies’, Journal of Management Information Systems 20(1), 123–151. https://doi.org/10.1080/07421222.2003.11045751

Hartung, J. & Knapp, G., 2005, Multivariate multiple regression, John Wiley & Sons, Ltd, Hoboken, NJ.

Industrial Development Corporation, 2012, Industrial Development Corporation predicts double-digit growth of IT spending by South African Government [Online], viewed 25 December 2014, from http://www.idc.com/getdoc.jsp?containerId=prZA23758612

Joseph, N., 2013, ‘A predictive model for information technology project success’, PhD thesis, University of Johannesburg, Johannesburg.

Koch, A.S., 2005, Agile software development: Evaluating the methods for your organisation, Artech House, Boston, MA.

Lindvall, M., Muthig, D., Dagnino, A., Wallin, C., Stupperich, M. & Kiefer, D., 2004, ‘Agile software development in large organisations’, Computer 37(12), 26–34.

Marnewick, C., 2012, ‘A longitudinal analysis of ICT project success’, in Proceedings of the South African Institute for Computer Scientists and Information Technologies Conference, ACM, Pretoria, South Africa, October 01–03, 2012, pp. 326–334.

Marnewick, C. & Labuschagne, L., 2009, ‘Factors that influence the outcome of information technology projects in South Africa: An empirical investigation’, Acta Commercii 9, 78–89. https://doi.org/10.4102/ac.v9i1.98

Misra, S.C., Kumar, V. & Kumar, U., 2006, ‘Success factors of Agile software development’, in Software engineering research and practice, pp. 233–239, Carleton University, Ottawa, Canada.

Misra, S.C., Kumar, V. & Kumar, U., 2009, ‘Identifying some important success factors in adopting agile software development practices’, Journal of Systems and Software 82(11), 1869–1890. https://doi.org/10.1016/j.jss.2009.05.052

Nguyen, D., 2016, ‘Success factors that influence agile software development project success’, American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) 17(1), 171–222.

Pallant, J., 2013, SPSS survival manual, McGraw-Hill Education (UK), London, England.

Pomykalski, J.J., Dion, P. & Brock, J.L., 2008, ‘A structural equation model for predicting business student performance’, Journal of Education for Business 83(3), 159–164. https://doi.org/10.3200/JOEB.83.3.159-164

Ramesh, B., Cao, L. & Baskerville, R., 2010, ‘Agile requirements engineering practices and challenges: An empirical study’, Information Systems Journal 20(5), 449–480.

Stankovic, D., Nikolic, V., Djordjevic, M. & Cao, D.B., 2013, ‘A survey study of critical success factors in agile software projects in former Yugoslavia IT companies’, The Journal of Systems and Software 86(6), 1663–1678. https://doi.org/10.1016/j.jss.2013.02.027

Taromirad, M. & Ramsin, R., 2008, ‘Cefam: Comprehensive evaluation framework for agile methodologies’, paper presented at the Software Engineering Workshop, 2008. SEW’08. 32nd Annual IEEE, Kassandra, Greece, October 15–16, 2008.

Tibenderana, P., Ogao, P., Ikoja-Odongo, J. & Wokadala, J., 2010, ‘Measuring levels of end-users’ acceptance and use of hybrid library services’, International Journal of Education and Development using Information and Communication Technology 6(2), 33–54.

Triandis, H.C., 1980, ‘Reflections on trends in cross-cultural research’, Journal of Cross-cultural Psychology 11(1), 35–58.

Venkatesh, V., Morris, M.G., Davis, G.B. & Davis, F.D., 2003, ‘User acceptance of information technology: Toward a unified view’, Management Information Systems Quarterly 27(3), 425–478.

Original Research

Modelling the critical success factors of agile software development projects in South Africa

Tawanda B. Chiyangwa, Ernest Mnkandla

Abstract

Introduction

Problem statement

Primary research question

Secondary research questions

Background

Methodology

Formulation of hypothesis

Performance expectancy

Organisation factors

People factors

Technological, process and project factors

Data presentation and analysis

Reliability, validity of the constructs and correlation

Reliability of the constructs

Convergent and discriminant validity and correlation

Structural equation results

Discussion of results

Performance expectancy

Success of agile software projects

Conclusion

Acknowledgements

Competing interests

Authors’ contributions

References