key: cord-0067812-at2mzcpc authors: Weekly, Charlotte title: Lessons in Failure: Applying an Organizational Learning Framework to Understanding Attitudes Towards Failure in Development date: 2021-09-20 journal: Environ Health Insights DOI: 10.1177/11786302211044348 sha: b6ce8aca36e672d050793dbf1af9b8a587a1eead doc_id: 67812 cord_uid: at2mzcpc This paper applies an organizational learning framework to explore attitudes towards failure in the water, sanitation and hygiene (WASH) development sector. It draws on 35 key-informant interviews, contextualized by organizational theory and existing scholarship on failure in development, to understand the challenges faced by WASH practitioners in identifying failure, analyzing failure, and deliberate experimentation. Through interrogating past and present initiatives for publicizing failure in development, this paper digs deeper into the successes, obstacles, and lessons learnt from mainstreaming failure into organizational practices. It then synthesizes these findings to advance a 3-tier conceptual map for organizations to build an enabling environment for learning from failure in development. It is no secret that development projects fail. Despite countless programs, billions of dollars invested, decades of debate and some success, huge numbers of people around the world are still living without access to water, sanitation, and hygiene services. 1 The idea that development practitioners should learn from failure has garnered considerable public support, yet organizations that systematize learning from failure are rare. 2 Even rarer, are studies that examine the effectiveness of attempts to mainstream failure into organizational practices. This paper seeks to contribute to the lack of knowledge in this area through following up years later with these initiatives; to dig deeper into the successes, obstacles and lessons learnt, to address the question of the "failure to learn from failure" in international development. 3 Despite several attempts to mainstream learning from failure into development practices, many of the most prominent initiatives, including the Engineers Without Borders Failure Report, the World Bank FAILFaire, and the WASHaholics Anonymous forum, have now been discontinued. Development authors have set out several accounts of why learning from failure poses such a challenge. Often cited are issues of accountability, most commonly, the power dynamic between donors and implementing agents. 4-6 Smillie 7 notes that "development enterprise is notoriously risk averse; donors demand [positive] results and punish failure." For implementing agencies, especially smaller non-profits with a heavier reliance on external funding, to survive in the current development landscape they must be seen to have strong operational efficiency and align their practices with donor interests. [8] [9] [10] Instead of acknowledging and addressing the failure, this can lead individuals to deny, distort, or cover up failures, and organizations and groups to suppress awareness of failures. Development organizations must also overcome technical barriers to identifying failure including causal ambiguity and system complexity. This is amplified in an aid environment where real-world development outcomes are notoriously difficult to evaluate. In the case of WASH services, there is no current accepted definition of "functionality," or what constitutes a functioning water point, which makes it difficult for researchers, governments, donors, and practitioners to understand the causes of failure. 11 The lack of real-world counter-factuals also poses a challenge, as well as the problem of attribution-that is, isolating causality. Beyond a limited number of cases where counter-factual analyses are possible through, for example, randomized treatment and control groups of sufficient size, it becomes particularly difficult to attribute success or failure to particular aspects of the project design, underlying conditions, or exogenous factors. 12 A further barrier to identifying failure in development is Chambers' 13 question of "Whose reality counts?" Stakeholders often exist in opposition to each other; projects considered a success in the eyes of the donor or implementer can be perceived to be a significant failure for the end-user. 12 Moreover, the outcomes of success or failure can be openly manipulated. Parker and Allen 14 point to an example where large-scale, donor-funded health programs were continued in Africa despite not working effectively. Public health officials initially claimed ignorance, however, when evidence later revealed widespread problems and weak uptake across the program, the façade of success continued to be upheld by officials through attempts to actively control and discredit contrary evidence. 15 Therefore, determining success or failure in development is less about objective indicators and is often a "negotiated truth" which is "found in the interpretation of events and actions." 16 Given this setting and the pressure on 'success', there is an incentive for perversely defining easier and more achievable goals to distort perceptions of the value of these projects. This can impede 2 Environmental Health Insights learning from failure where the information shared is subject to rigorous editing and narrative construction. Social system barriers also pose a significant challenge for identifying failure. "Learning from failure is not automatic or instantaneous," but instead complicated by a wide range of interpersonal and intrapersonal factors. 17 Failure often triggers negative emotions which inhibits ability to reflect on what went wrong, hinders cognitive processes and limits consideration of other alternatives. [18] [19] [20] When negative emotions such as anxiety or grief consume most cognitive resources, individuals struggle to reflect on failure and analyze relevant information, which is crucial to the learning process. 21 This is applicable across all sectors, but particularly true of development which is emotively driven work. 22 Furthermore, the resistance to analyzing failure in development may be linked to the tendency of organizations to look forwards rather than backwards. Lewis' 23 work on the "perpetual present" of policymaking is relevant here. The world of policymaking is one where the past is constantly being repressed. Organizations often look forward to what can be better in the future and a never-ending focus on novelty and change has come to characterize the international aid environment, which does not lend itself to productive analysis of past failures, and in many cases, can be the cause of them. A recent movement has called upon organizations to go 1 step further and "fail forward" in international aid practices. [24] [25] [26] Fail forward culture, which emerged from private sector and Silicon Valley thinking, calls for active risk-taking, to fail fast, fail often, and implement these learnings to grow and innovate. Silicon Valley investors regularly reward entrepreneurs for risktaking, despite knowing the venture could fail and they could lose their capital. 27 Some authors have cited the success of fail forward culture in venture capitalism, engineering, and design, and argue that development organizations, particularly donors, should emulate Silicon Valley's culture of calculated risk-taking. 40 However, few studies have thoroughly examined whether fail forward is a useful methodology in development or simply the latest example of trying to follow the private sector. In order to test this, the following framework has been identified from the business world. Cannon and Edmondson's 2 framework identifies 3 distinct but interrelated processes for organizational learning from failure: (1) identifying failure, (2) analyzing failure, and (3) deliberate experimentation. While this framework was designed for the private sector, the key processes around learning from failure and many of the social and technical barriers are applicable to the international aid landscape. Further, using a business-derived framework brings a new perspective in the context of increasing calls to apply private sector thinking around deliberate experimentation to international aid practices. The upper level of the framework describes the technical system barriers and makes recommendations for mitigating these including training, education, and technical expertise, while the lower level of the framework discusses the social system barriers and makes recommendations for building psychological and organizational capabilities to engage in identifying failure, analyzing failure, and deliberate experimentation ( Figure 1 ). The following section considers Cannon and Edmondson's 3 processes of learning from failure: identifying failure, analyzing failure, and deliberate experimentation. A review of current and past initiatives is conducted, guided by the framework, to discuss the barriers faced by aid practitioners and the opportunities for overcoming these challenges. 35 key-informant interviews were conducted between June and August 2020 as part of this research. Qualitative, in-depth interviews were selected as the most appropriate method to gain deeper insight into the subject, as they go beyond surface level and allow for a rich description of processes. 28, 29 Purposive sampling was used to ensure interviews encompassed a wide range of perspectives across various organizations. 30 Key-informants were identified from current and former staff members, based on their ability to speak to the organizational decision-making processes. Snowball sampling was used to reach key-informants that might otherwise have been hard to access. 31 Rather than aiming to achieve a fully representative sample across the population, the goal was to select the key-informants that would be able to give the most insight into the research question. 32 Semi-structured interviews gave participants the freedom to answer the questions according to what they thought was most important. 33 A guide was developed for the interviews but remained adaptable to emerging themes; in this way, an iterative and inductive study was established rather than researcher imposed. 34 With participant approval, the interviews were audio-recorded then transcribed verbatim to allow full immersion in the data. 35 The interviews were analyzed thematically in order to synthesize and interpret the data. Thematic analysis was used for its flexibility as a research method for finding patterns and meaning across qualitative data. 36 Braun and Clarke's 37 6 stage framework was followed to structure the thematic analysis. This involved re-reading the interview transcripts multiple times to search for "recurring regularities." 38 Interesting features in the data set were systematically coded manually and categories of meaning were identified and organized into themes. The purpose of these "themes" was to represent areas of importance and patterned meaning across the data to shed light on the research question. All data was organized in accordance to emerging themes, and all codes and themes were adapted throughout to ensure an iterative approach. Given the focus of this paper on examining organizational approaches to learn from failure in development, the names of Weekly 3 participant organizations have been retained to provide important context to the area on which they are speaking. There are ethical concerns that come with revealing any aspect of participant identity and great care was taken to not put any individuals at risk. To ensure confidentiality is protected, job titles have been anonymized throughout. Meaningful informed consent was collected from all key-informants through signed consent forms. Participants were reminded at the start of the interview of their right to remain anonymous, which was re-iterated later if any sensitive issues came up during the discussions. Additionally, before presenting any findings that might be controversial, interviewees were consulted to ensure they consented to the quote being used. All digital files, transcripts, and summaries were given codes and stored separately from any names or other direct identification of participants. Any hard copies of research information were kept in locked files at all times. The decision to identify participant organizations, and the approach to ensure participant safety was developed in consultation with the LSE Ethics Committee. This section presents the findings and major themes emerging from the key-informant interviews. The first section examines the barriers for (1) identifying failure, (2) learning from failure, and (3) deliberate experimentation. The second section analyses key factors to build an enabling environment for organizational learning from failure. The third section draws together these themes to develop a framework for the development sector. Barriers to learning from failure in development Identifying failure Not all failures are created equally. When examining the barriers to identifying failure within organizations, a major theme that emerged was how different the experience of failure is. Throughout the key-informant interviews, participants identified a myriad of factors, including race, gender, age, geographic location, economic circumstances and ranking within organizations, that posed significant barriers to discussions of failure. A CEO of a failure consultancy firm spoke on intersectional aspects of failure: "It is one thing for me to sit here as a privileged white woman. . . to say, 'yes of course we should all talk about failure'. It is another thing for someone where, when they fail, it is assumed it is because of their incompetence. . . It is so much harder for people of color or minorities to [go through the process of failure]. We need to recognize that the experience of failing and how you're treated after is very different." Key-informants noted that identifying failure might be more challenging in the Global South, given the inequitable structures embedded in North-South relations: "There are inequitable structures in place that would make raising failure very risky for people that have less financial or political power. . .. Instead of being seen as pivoting or innovation, it's seen as well, you didn't fulfil a mandate." (Founder of a social equity enterprise). Other participants made the link between job security and the ability to raise failure: The language of failure. Failure is a taboo word not just in development but across cultures. Key-informants noted that it may not always culturally appropriate to talk about failure: "In different countries, you have to adjust your language differently to ask them about failure in a way that is going to be acceptable within the current culture. For some people that is removing the word failure entirely; 'things that have gone wrong', 'projects that have been less successful that planned', 'lessons learnt.'" (Co-founder of a WASH failure pledge). Other participants discussed the socio-cultural dimension of the language of failure: The language of failure can also be politically sensitive. A WASH Manager at UNICEF highlighted this: "In organizations such as the UN, we rebrand it as lessons learnt and best practices which is less aggressive than talking about failure." Failure for whom?. When discussing attempts to analyze failure within development for organizational learning, several key-informants raised concerns over whose voices have been centered in this discourse. Speaking on the failure report, A former employee at Engineers Without Borders Canada, questioned why the voices of end-users were not included: A Senior Manager at CARE USA echoed this concern of a "blame game" within organizations: Throughout the key-informant interviews, many participants highlighted the need to create an "enabling environment" where individuals feel not only safe but rewarded for learning from failure. When asked to unpick what that might look like in practice, several key themes emerged. Foundational level. Creating this culture begins at the foundational level. Many participants highlighted the importance of "psychological safety" in an organizational environment, where individuals feel safe to take risks and not be punished or shamed for speaking up with ideas, questions, concerns, or mistakes: "If there isn't that culture there already, where I feel like I will be fired if I make a mistake or ostracized or punished rather than helped to use it as a learning moment, then I would not share any story no matter how institutionalized it is." (Senior Official at Engineers Without Borders Canada). Representatives cited the importance of having a growth mindset and culture of learning embedded from the organization's inception: "BRAC has always been a learning organization. . . the culture of piloting and if something doesn't work, pivoting very quickly. . . Having that legacy within the organization as something to reference was a huge enabling factor." (Former employee at BRAC Social Innovation Lab). "We were founded by two university students who were engineers. . . young, energetic, hopeful people who were untraditional and unconventional in their approach to sector. . . We had the benefit of setting this culture early on." (Senior Official at Engineers Without Borders Canada). Key-informants also cited the importance of having a space for controlled failure within the organization to promote deliberate experimentation and learning: "We have a license to fail as we were set up as an experimental unit. We were set up with the aim to. . . be more adaptive, more agile, try out more ideas to see what works. That's a huge freedom." (Employee at BRAC Social Innovation Lab). Skills and behaviors. The next stage is building the skills and behaviors within organizations to have productive conversations about failure, then institutionalizing and sharing the learning so mistakes are not repeated. Across the key-informant interviews, many participants stressed the role of agile and adaptive leadership in this: "The biggest support we enjoy is the inspiration from the founder and leadership who embody or try to build a culture of innovation throughout the organization." (Employee at BRAC Social Innovation Lab). A key motivating factor for driving conversations about failure is employees being able to see examples of previous failures being incorporated into program iterations and design of new solutions: "The biggest incentive I have found for anyone to share their experience is the belief that something will change because of it. . . It comes back to the feedback loop, the reason you would take a risk is because there is a reward. And the reward is that we change and we get better." (A Senior Manager at CARE USA). "The problem with lessons learnt is that they have become something superficial and not productive. . . People need more assurance that something will change in your organization." (A Senior Manager at FuckUp Nights). Key-informants emphasized the importance of structural incentives for learning from failure within organizations: "Twenty percent of my performance objective was to start something within the organization that had never been done before. . . That was a huge enabling factor." (Former employee at BRAC Social Innovation Lab). At the top level are the formal mechanisms, structures, and practices that embed a culture of learning from failure within organizations. Participants noted the importance of ensuring these messages are reinforced across organizational practices: Throughout the interviews, several participants highlighted organizations must invest in promoting a narrative around learning from failure: "Most entities underinvest in the actual narrative around the innovation, the failure and the learning. . . One of the best things an organization can do is be really intentional about making its journey digestible. . . Not only that but celebrating the people who take risks and the people who fail." (A former Senior Manager at USAID). At the highest-level, relationships must be nurtured with donors that are built on transparency and trust: "Psychological safety, trust and a good relationship between the donor and the implementer is essential for being able to talk about [failure] ." (Former Advisor to DFID). "We need it to be a norm that every report that goes to a funder talks about failures as well as successes, every policy brief should have a section that talks about failure." (Co-founder of a WASH failure pledge). Consideration of the social and technical barriers to organizational learning from failure explored in the literature, alongside the lessons learnt and reflections gathered from the keyinformant interviews, has resulted in the following framework designed as part of this paper that maps the infrastructure needed to build an enabling environment for learning from failure in development (Figure 2) . At the foundational level, organizations must ensure psychological safety, where all team members feel safe to take risks and be vulnerable. 39 What this looks like in practice differs across organizational contexts, but key-informants highlighted the importance of job security, diversity, inclusion, and strong encouragement and empowerment coming top-down from leadership. Psychological safety must be positioned alongside a growth mindset, honesty, transparency, and a commitment to ongoing self-awareness and evaluation throughout the organization. This is most effective when a culture of learning is embedded from the organization's inception, as has been seen in the case of BRAC and Engineers Without Borders Canada. The development literature also highlighted the importance of establishing a "safe space" for failure, for promoting deliberate experimentation and testing new ideas in a controlled environment. 40, 41 For this to take place, organizations must ensure staff Weekly 7 and volunteers have the language to speak about failure. This needs to take place in a culturally sensitive way with the understanding that the word "failure" may not always be appropriate. "Find voices, local voices, Southern voices, that understand the value of failure, and they can speak in a language or deliver it in a language that people can easily adopt. . . It has to be driven by local voices and customized for the local context." (Managing Director of a WASH non-profit). Developing the skills and behaviors to have productive conversations about failure is paramount. Organizational training, adapted to fit the local context, would be a beneficial tool for overcoming the psychological barriers to analyzing failure in order to enable learning. Alongside training, staff and volunteers must be able to see concrete examples where failure has not only been acknowledged but incorporated into program iterations and the design of new solutions, which operates in the service of the end user. In this way, risk-reward mechanisms can be established. Part of this is also building in incentives for learning from failure, which could take the form of key performance objectives, as experimented with by BRAC. Another crucial aspect highlighted by participants was the importance of agile, adaptive, and forward-thinking leadership, with the commitment to drive a vision and within that to promote learning from failure. "The commitment of leadership is really important because it allows for role modeling, setting expectations clearly, for leaders to have understood what is behind a culture At the highest-level, commitment to learning from failure must be formalized through organizational structures and monitoring and evaluation systems. The way organizations hire, train, and conduct meetings, evaluations and performance reviews are crucial for embedding this culture within institutional structures. "People have to feel ownership over [failure culture] like it represents them. The reason why it is such an effective cultural tool is that people take it on as an identity." (CEO of a failure consultancy firm). In addition to this, participants noted organizations must invest in internal and external communication to build a narrative for learning from failure. Perhaps most importantly, across both the literature and key-informant interviews, the value of building strong relationships with donors based on transparency and trust was emphasized. Donors must encourage deliberate experimentation through creating a culture that is not only tolerant but rewarding of failure. 6, 40 As part of this, there must be external accountability mechanisms and neutral watchdogs to hold donors accountable. One possibility could be score cards where donors are marked on the extent to which they have looked back on past projects they have funded, to see what has happened and taken steps to implement these learnings, which can work to shift behavior. It is important to acknowledge, too, that these events do not take place within a vacuum. Exogenous factors have a large influence on organizational environments and the ability to learn from failure. Competition for funding in development is heightened following the 2008 financial crisis and the ongoing Covid-19 pandemic has only brought further economic instability. Resources directed to the development sector are shrinking and donors, governments and international NGOs are under increased scrutiny to deliver positive results. Great care must be taken to emphasize the learning aspect of failure, rather than the failure itself, to avoid being seen to celebrate failure at a time when the lives and wellbeing of marginalized people around the world are at risk. The framework integrates the key challenges in both the development literature and the informant interviews, with recommendations for overcoming the social and technical barriers for learning from failure. If properly applied, this framework can lead to productive conversations on failure that enable organizational learning and new development solutions to complex problems. However, this framework is positioned as a learning resource, not a mandatory approach or a tick-box activity. Organizations can be encouraged to learn from failure in development, but as we have seen, if this is made mandatory, the discussion can become performative and less productive. Participants were selected from a wide range of organizations: NGOs, IGOs, donors, private sector and civil society, with a focus on WASH development policy-makers and practitioners, all involved in systematizing learning from failure in international development. However, the sample generated was dependent on the participants that responded to requests for interview. There is also a possibility of result-bias where interviewees may have been reluctant to speak negatively on the practices of their current organization. It was noted that former employees were generally more open to reflect on lessons learnt than those that remained in employment at the organization they were speaking to. To overcome this, participants were assured of anonymity during interviews which was re-iterated later if any sensitive issues came up during the discussions. This paper has sought to explore attitudes towards failure in WASH development through examining the social, technical, and organizational challenges facing practitioners in identifying failure, analyzing failure, and deliberate experimentation. The major themes emerging from the key-informant interviews and the literature on organizational learning were synthesized to draw the following conclusions. First, the notion of safety must be expanded. Despite Cannon and Edmondson highlighting the importance of psychological safety for having conversations about failure within organizations, the literature underestimates the structural inequalities that constrain individual agency. Actors' positionality affects not only their experience of failure but their capacity to identify it. A myriad of factors including gender, race, age, geographic location, economic circumstances, and ranking within organizations intersect to constrain the ability of actors to engage in conversations about failure. This must also be contextualized within an international aid architecture which has a long-standing history of paternalism and colonialism. Furthermore, across contexts, the language of failure has different socio-cultural implications. Given these considerations, it is critical that discussions around failure take place in a culturally sensitive manner, driven by local voices, and adapted for the local context. Learning from failure must be encouraged, as opposed to mandated from the top-down, or from the Global North to the Global South, to avoid compromising individuals' need for safety. Second, the question of "Whose reality counts?" was raised in both the literature and throughout the key-informant interviews. While conversations about failure can be a useful tool for promoting the voices of service-users, it is less useful when it is the implementing agency that is defining the failure. Several participants voiced these concerns and warned that neglecting to center the end-users in these discussions has resulted in performative commitments where organizations are unwilling to analyze failure beyond the surface level. This is symptomatic of a wider trend of expert bias within development. To overcome this, organizations must become customer centric, drawing lessons from the private sector on accountability, to center service-users' voices before learnings from failure can be conducted successfully. The learning aspect of failure must be the focus, Weekly 9 rather than the failure itself, to avoid performative commitments from development organizations. Third, the role of donors and deliberate experimentation was highlighted across the development literature and the interviews. Donors must shift their priorities to encourage deliberate experimentation through creating a culture that is not only tolerant but rewarding of failure. As part of changing the way they operate, donors must take steps to incentivize learning from failure in the projects they fund. While representatives from the World Bank and DFID expressed reservations throughout the key-informant interviews, perhaps donors must consider why they are reluctant to confront failure and what this says about their own practices. Notable, too, was a certain degree of "passing the buck" that occurred across the key-informant interviews. NGOs would argue that it was the role of donors to drive learnings from failure, and donors would counter that it was the role of NGOs. This reveals much about the embedded nature of the problem. "In the end, who pays decides." (WASH Manager at UNICEF). However, organizations cannot afford to be complacent and system-wide advocacy approaches are needed across the development sector and civil society. Ultimately, there is no single solution for these challenges. However, what emerged from the literature and the keyinformant interviews was the need to build enabling environments within organizations to institutionalize learning from failure. Drawing on these themes, a framework was designed as part of this paper which, if applied correctly, can lend itself to productive conversations about failure. As this paper has discussed, organizational learnings from failure require a high-level of commitment, resources, time, and energy. Yet, the pay-off is high. The Covid-19 crisis has brought with it a wave of new challenges for the international aid system, with development practitioners and policymakers are encountering problems that have never been seen before. 42 In this context, it is more important than ever to innovate, iterate, and improve solutions to development challenges. Through harnessing lessons from failure, we can overcome these global challenges, and many more to come in the future, to find new solutions for tomorrow. Charlotte Weekly https://orcid.org/0000-0002-3904-6968 Engineering and humanitarian intervention: learning from failure Executive summary: failing to learn and learning to fail (intelligently): how great organizations put failure to work to innovate and improve 2010 failure report: learning from our mistakes Going Global: Transforming Relief and Development NGOs Keeping body and soul together: southern NGOs' strategic responses to donor constraints NGOs, states, and donors revisited: still too close for comfort? Learning, development, and money: lessons from Bangladesh Contests, NGOs, and decentralizing aid The international aid system and the non-governmental organisations: a new research agenda Beyond 'functionality' of handpump-supplied rural water services in developing countries Ineptitude, ignorance, or intent: the social construction of failure in development Whose Reality Counts? Putting the first Last. Intermediate Technology Publications De-politicizing parasites: reflections on attempts to control the control of neglected tropical diseases Strategic unknowns: towards a sociology of ignorance The social construction of development cooperation success and failure Learning from business failure: propositions of grief recovery for the self-employed Confronting failure: antecedents and consequences of shared beliefs about failure in organizational work groups The role of affect in the entrepreneurial process New Approaches to Leadership, Cognitive Resources and Organizational Performance The Oxford Handbook of Organizational Decision Making Editorial: emotional methodologies-the emotional spaces of international development International development and the 'perpetual present': anthropological approaches to the re-historicization of policy The F-word of the development world Failing forward: how to encourage sharing of what is not working. Overseas Development Institute To do better research on the global South we must start failing forward Failing by design Qualitative Research Interviewing: Biographic Narrative and Semi-Structured Methods. SAGE Handbook of Interview Research Encyclopedia of Survey Research Methods. SAGE Sampling, snowball: accessing hidden and hard-to-reach populations Interpreting Qualitative Data. SAGE Qualitative Research for Education: An Introduction to Theories and Methods. Pearson Education Group Qualitative Interviewing: The Art of Hearing Data. SAGE Introducing Qualitative Research in Psychology Encyclopedia of Case Study Research Using thematic analysis in psychology Qualitative Research and Case Study Applications in Education The local and variegated nature of learning in organizations: a group-level perspective Learning From Failure: Lessons for the Sanitation Sector. UK Sanitation Community of Practice Development Projects as Policy Experiments: An Adaptive Approach to Development Administration. Taylor and Francis OECD. Innovation, development and COVID-19: challenges, opportunities and ways forward. 2020. Accessed