|
Orientation: Measuring the target outcomes of leadership development programmes provides evidence for the effectiveness of these interventions and the validity of their theoretical underpinnings.Research purpose: The aim of this study was to determine whether staff from the Tasmanian Department of Health and Human Services (Australia) experienced increased levels of self-efficacy, social support within the workplace and positive affect, following participation in a leadership development programme. Research design, approach and method: Quantitative and qualitative methods were used, allowing for triangulation of results. The General Self-Efficacy Scale and the Berlin Social-Support Scale (perceived available support, instrumental) were applied in an online survey administered before and nine months following the programme. Participant satisfaction surveys captured immediate responses and semi-structured interviews captured longer-term reflections. Main findings: Descriptive statistics indicated a moderate overall increase in self-efficacy, with strong increases in resilience, dealing with opposition, resourcefulness and problem solving. There was some evidence of greater overall social support and a strong increase in the development of social support networks. There was no support for an increase in participants’ positive orientation towards their jobs in the quantitative data. The impact of adverse environmental factors on participants’ perceptions also became evident through the interviews. Practical implications: Leadership development programmes that strengthen positive psychological resources provide participants with confidence and resilience in times of change. Organisations benefit from increased levels of employee self-efficacy as engagement and problem-solving abilities are enhanced. Contribution/value-add: These results contribute to the body of knowledge associated with effective leadership development.
Workplace leadership development programmes represent a significant investment by employing organisations. Evidence of effectiveness is required in order to justify this investment against other, competing, priorities. This is particularly so in the area of public sector health services provision, where limited budgets and unmet demand combine to create organisational cost pressures.
Background to the study
Health services provision is costly, and that cost is rising at an unsustainable rate across the globe (Organisation for Economic Co-operation and Development [OECD], 2011; World Health Organization [WHO], 2010). Decisions about the provision of health services are typically made by professionals with considerable authority and autonomy (Braithwaite, Skinner & Doery, 2011). The United Kingdom, Canada and, more recently, Australia have invested in public sector health services leadership development as one strategy for fiscal sustainability (Dickson, 2007; Health Workforce Australia, 2013; NHS Institute for Innovation and Improvement, 2010). In Australia, one example of a health leadership programme is the Tasmanian Department of Health and Human Services (DHHS) Leadership and Management Development Programme. It consists of three components: a development programme for front-line and aspiring managers, an academic programme delivered with the University of Tasmania and a graduate trainee recruitment programme (Shannon & Burchill, 2013). The focus of this paper is the development programme, which involves a multi-day structured off-site course and a series of workplace activities, including participation in an action learning group, a coaching relationship, two shadowing opportunities and a workplace project. Both course and workplace activities relate to five focus areas that reflect themes common to other health leadership frameworks (Dickson, 2007; Health Workforce Australia, 2013; NHS Institute for Innovation and Improvement, 2010) as illustrated in Table 1.
|
TABLE 1: Main components of health leadership frameworks compared.
|
The Tasmanian health leadership framework themes were incorporated into the development programme in the following manner:1. Strategic Focus presentations provide context and organisational awareness, a greater understanding of strategic directions and current challenges, with an opportunity to directly engage with senior leaders. This is reinforced by the workplace activity of shadowing. 2. Achieving Results presentations provide factual information about finance and human resource management systems. Workplace projects are often aligned to this focus. 3. Quality and Governance presentations provide national and policy context for local work to improve the safety and quality of services. Coaching relationships also provide a quality assurance function during the workplace activities. 4. Managing People presentations provide context and organisational awareness that enables participants to reflect on their management practice and team dynamics. Action learning groups also often focus on this theme. 5. Leadership and Self-development presentations share leadership stories and examples and provide preparation for post-course activities. A causal theory associated with positive organisational behaviour (Luthans, 2002a; 2002b) was adopted to underpin the DHHS development programme, aligned with the self-assessment instruments used by participants during the off-site course. The Competing Values Management Survey (Edwards, Austen & Altpeter, 1998; Quinn, 1988), the Organizational Culture Survey (Quinn, 1988; Van Beek & Gerritsen, 2010) and the Fundamental Leadership Survey (Quinn, 2005) formed the backbone of participant self-assessment of their individual management and leadership styles and their interaction with DHHS organisational culture. As positive organisational behaviour is developmental, dealing with psychological ‘states’ rather than ‘traits’, this self-assessment is posited to be open to revision through changes in three sets of resources: agentic, relational and affective (Roberts, Dutton, Spreitzer, Heaphy & Quinn, 2005). Agentic resources include a sense of self-efficacy and an ability to positively envision oneself in the future. Relational resources are the result of positive interpersonal relationships that provide social and emotional support within the workplace. Affective resources include all positive emotions and, in the work setting, include positive orientation towards the job (Roberts et al., 2005). Changes in these resources, over time, provide target outcomes for evaluation. The evaluation of the DHHS leadership development programme was based on the concepts behind these self-assessment instruments.
Research purpose
Few leadership development programmes have an explicit causal theory – an explanation of how and why the development activities would lead to outcomes and impacts – and most evaluate short-term outputs only, such as participant numbers and satisfaction with the programme (Russon & Reinelt, 2004). Literature providing a meta-evaluation of workplace leadership development programmes (Development Guild/DDI, 2002; Hannum, Martineau & Reinelt, 2007) provides guidance for new programmes but does not provide a single preferred method. Instead, Hannum et al. (2007, p. 4) suggest that: ‘different leadership development programs, different organizational or community contexts, and different evaluation questions demand a broad variety of approaches’.The aim of this evaluation was to determine whether DHHS staff experienced an increase in their agentic, relational and affective resources following participation in the DHHS development programme. The alignment of the DHHS development programme with positive organisational behaviour and the work of Roberts et al. (2005) led to a search for associated (quantitative) evaluation instruments. The evaluation methodology most associated with positive organisational behaviour, the Psychological Capital Questionnaire (PsyCap) (Luthans, Youssef & Avolio, 2007a), shares an interest in self-efficacy and an optimistic view of the future. Whilst addressing affective resources in terms of seeing hope as a key concept, PsyCap does not explicitly include relational resources in the remaining key concept of resilience (Avolio, Luthans & Youssef, 2007). An examination of those leadership programmes that had used the advanced levels of evaluation associated with the Kirkpatrick evaluation framework (Kirkpatrick, 1994, cited in McLean & Moss, 2003) indicated an ‘inverse relationship’ between effort and results. The ‘substantial costs’ associated with this form of leadership programme evaluation (McLean & Moss, 2003) made it unsuitable for evaluation of the DHHS development programme. The Leadership Programme Outcomes Measure (Black & Earnest, 2009), associated with the EvaluLEAD framework (Grove, Kibel & Haas, 2005) also presents an expanded evaluation agenda, requiring the measurement of ‘community outcomes’ of leadership development. In order to adequately capture the change in the target outcomes of the evaluation, in a manner that could be undertaken within existing resources, it was necessary to develop a leadership development programme evaluation tool. It needed to be suitable for use in the health and human services sector and extend beyond short-term satisfaction measures to capture outcomes over time.
Contribution of the study
Through the development of a leadership development evaluation tool explicitly linked to the work of Roberts et al. (2005), the results of this study contribute to the body of knowledge associated with positive organisational behaviour. To the extent that this study presents the results of an evaluation over time, it contributes to the understanding of workplace leadership development programmes. The placement of this work in health and human services is significant for leadership development in the sector. The rest of this article describes the contextual environment, research design, responses, results and findings of the development programme evaluation. It then discusses the implications these have for this and other leadership development programmes.
Literature review
Health services: A challenging fiscal environment
One of the most significant challenges facing the delivery of health services is the increasing cost of care. International comparisons of health care cost are commonly described as the percentages of gross domestic product (GDP) taken up by total health expenditure. Comparisons between countries within the OECD show health spending growth has surpassed GDP growth by almost two to one over the last decade (OECD, 2011). Due to the global economic recession, this trend has increased since 2008 and has increased pressure on health service providers to eliminate ‘wasteful spending’ and demonstrate value for money (WHO, 2010).Within Australia, public hospitals, medical services and pharmaceuticals make up the majority of health-related expenditure, whilst the national government provides the majority of the funds (Australian Institute of Health and Welfare, 2012). Prior to 2012, the majority of funds were provided to state and territory governments, to be allocated to hospitals and other health services. These historically based allocations had not kept up with demand or cost increases. In 2012, through the National Health Reform Agreement, the Australian government initiated a major restructure of the health system and the introduction of new ‘activity-based’ funding arrangements (Council of Australian Governments, 2011). In Tasmania, as in many other states and territories, government expenditure has been increasingly limited. In recent years this resulted in organisational downsizing within publicly funded health services of approximately 10% (O’Byrne, 2011; 2012). An environment of organisational downsizing and restructuring has been shown to have negative impacts on workplace physical and psychological health (Emmerik & Euwema, 2008; European Expert Group on Health in Restructuring, 2009; Lee & Teo, 2005; Swanson & Power, 2001). The link between cost issues and health services leadership works two ways, both cause and effect. Whilst health service providers are not primarily focused on financial matters (Shannon, Holden & Van Dam, 2012), they are significant drivers of cost in that they determine what procedures and equipment is required for patient treatment (Kosimbei, Hanson & English, 2011). Health service providers are also impacted by cost reductions associated with organisational downsizing and restructuring (Shannon, Van Dam & Stokes, 2012). The distributed model of leadership active within health services provides a challenging context for the organisation.
Health services leadership development and positive organisational behaviour
The ‘fit’ between health services leadership development programmes and positive organisational behaviour is the link with self-efficacy and resilience, the proactive and reactive responses of the individual within a workplace context (Luthans, 2002a). On one hand, individuals who work as health professionals are expected to be highly qualified, with considerable expertise and experience. Their work requires the ability to successfully act on their environment (Braithwaite et al., 2011). On the other hand, a higher level of responsibility, and the constant ‘crisis’ in health services, due to fiscal and other pressures, flag the need for resilience. The study of positive organisational behaviour provides both understanding and strategies in a less-than-positive environment (French & Holden, 2012).The concept of positive organisational behaviour is particularly significant for health services. The contemporary model of health services leadership is that of a ‘distributed’ (Bolden, 2011) or ‘post-heroic’ leadership style, which recognises that leaders exist at all levels and in all occupational streams within an organisation. In this context, systematic leadership development across the organisation results in an ‘empowered organisation’ (Hancock & Campbell, 2006). It is in this sense also that the difference between ‘positive organisational behaviour’ (with its ‘micro’ focus on individual development) and ‘positive organisational scholarship’ (focused on ‘macro’ organisational issues) (Luthans, Youssef & Avolio, 2007b) becomes less relevant.
Research approach
A mixed-method approach was taken, drawing inferences from quantitative and qualitative data in order to explore, examine and understand the contextual world of leadership development (Doyle, Brady & Byrne, 2009; Leech & Onwuegbuzie, 2009). Whilst mixed-method research is becoming recognised as the third major research approach along with qualitative research and quantitative research (Johnson, Onwuegbuzie & Turner, 2007), it is a broad category that can include a range of research strategies. The authors followed a mixed-method approach that involved triangulating the descriptive data that emerged from questionnaire surveys and semi-structured interviews (Kvale, 1996). The qualitative research component of this study was based in constructivist grounded theory (Charmaz, 2006). This approach allows participant reflections on personal experience to shape the explanatory categories that are used to analyse that experience. In this study, open-ended questions within semi-structured interviews provided a participant ‘reality check’ for the quantitative results emerging from closed-question survey data (Jick, 1979). The authors are ‘pragmatic researchers’ (Onwuegbuzie & Leech, 2005), who rely on their research findings to inform their delivery of leadership and management development programmes.
Research method
A participant satisfaction survey captured initial responses to individual speakers and topics presented at the development programme course, as a form of process evaluation. The outcomes evaluation of the DHHS leadership development programme involved an online evaluation survey administered twice: initial (T1) data was collected two weeks before the development programme course and subsequent (T2) data was collected nine months after the course, when most participants would have completed all the workplace activities. In addition, a small number of interviews were undertaken between one and two years after the development programme course, to provide longer-term reflections on the impact of the programme.
Participants
The evaluation target population were participants in the development programme courses held between February and November 2011 (n = 205). These were broadly reflective of the gender ratio of DHHS, with a predominance of female employees (Department of Health and Human Services, 2012), and employees classified under the Administration and Clerical Workers Industrial Award. There was some over-representation of Allied Health professionals amongst development programme participants. This breakdown is shown in Table 2 and Table 3. The pre-course baseline outcomes survey (T1) attracted a 65% response rate (n = 134), the post-course participant satisfaction survey was filled out by 118 participants (58%), and the nine-month outcomes survey (T2) had a 42% response rate (n = 87). Men were less likely to complete the nine-month outcomes survey than were women.
|
TABLE 2: Gender of the Tasmanian Department of Health and Human Services staff, participants.
|
|
TABLE 3: Professional profiles of Department of Health and Human Services staff, participants.
|
Ethical considerations
All participants in the evaluation participated voluntarily. Information about the research was provided to all participants in the development programme and it was up to the participants whether they wished to ‘opt in’ to the research. Responses to the quantitative data gathering were anonymous. Qualitative data (interviews) were de-identified. Electronic copies of interview and survey data were kept in a password-protected folder. Data gathering was approved by the Human Research Ethics Committee (Tasmania, Australia) Network, permit number H12103. Purposive sampling was used to select interview participants who had completed the development programme and reflected the gender and professional background of DHHS staff and development programme participants. Potential participants were emailed information about the evaluation and asked if they would like to take part. If they agreed, a meeting room within the DHHS precinct provided an interview setting. Of the 35 participants who were initially approached, nine participants agreed to take part. Table 4 presents the biographical data of interview participants.
Measuring instruments
Participant satisfaction survey for immediate responses: A participant feedback form was provided at the start of each development programme course so that responses to presentations and exercises could be recorded in ‘real time’ or at the end of each day. Participants were requested to transfer their anonymous responses to the online questionnaire the week after their participation in the course. Whilst it was possible for participants to alter their paper responses when transferring them to the online questionnaire, they were requested to simply provide those initial reactions.The purpose of the survey was to gain participant opinion on whether the agenda of the development programme course had met their needs and to provide feedback to the senior executives who had spoken as presenters. Of particular interest to the evaluation, however, were participants’ free-text comments as to what aspect of the course they found most useful and their level of agreement with the statement that ‘the development programme course will be of value to my management and leadership practice’ in (1) their current job and in (2) their future career. Here respondents were given a choice of ‘strongly disagree’, ‘disagree’, ‘agree’, ‘strongly agree’. Outcomes evaluation survey for medium-term responses: The outcomes evaluation survey incorporated the 10-item General Self-Efficacy Scale (GSES) (Schwarzer & Jerusalem, 1995) and the four-item Berlin Social Support (Instrumental Perceived Available Support) Scale (BSSS) (Schulz & Schwarzer, 2003), together with one question relating to positive job orientation. The response options for each question of ‘not true’, ‘hardly true’, ‘moderately true’ and ‘exactly true’ in the T1 and T2 surveys were assigned numeric values of 1, 2, 3 and 4 respectively. Commercially available online software was used to administer the outcomes evaluation survey in a way that ensured anonymity of responses. An email was sent to participants requesting their participation in the survey and giving the URL of the survey website. Two weeks later, another email was sent to participants, reminding them of the survey availability. The GSES has been validated in a wide range of contexts (Liu & Lee, 2012; Scholz, Dona, Sud & Schwarzer, 2002) and has demonstrated internal consistency (Cronbach’s alpha values of between 0.75 and 0.91). This study sample resulted in a Cronbach’s alpha of 0.81. The sum mean for the GSES in this study was 32.54 with a standard deviation of 3.15. Studies with larger and more diverse populations have reported sum means ranging between 23.52 and 33.44 with a standard deviation of 4.00 to 5.42 (Luszczynska, Scholz & Schwarzer, 2005; Schwarzer, Bäßler, Kwiatek, Schröder & Zhang, 1997). The BSSS measures the level of tangible (material, physical or mental) support that the respondents anticipate will be available to them, should they require it (Schwarzer & Leppin, 1988). Previously, the subscale has shown a Cronbach’s alpha of 0.83 (Kapikiran & Kapikiran, 2010; Schulz & Schwarzer, 2003); for this study the Cronbach’s alpha value was 0.79. Sum means were not published. The evaluation questionnaire used in this study, made up of the two instruments described above combined with an additional question on job satisfaction, show an internal consistency of 0.83 and standard deviation of 4.54. These descriptive statistics are provided in Table 5.
|
TABLE 5: Descriptive statistics for outcomes evaluation questionnaire subscales.
|
Participant interviews for longer-term reflections: The qualitative research set out to evaluate changes in leadership and management behaviours after participation in the DHHS development programme through a series of semi-structured interviews. The timing of these interviews was determined by the availability of a graduate trainee, a qualified social worker, who was trained to undertake the interviews as part of her work. Temporarily attached to the programme team, she was able to gather data that was not influenced by the presence of established programme staff.The interviews were organised around a set of pre-determined open-ended questions, with other questions emerging from the dialogue between interviewer and interviewee (DiCicco-Bloom & Crabtree, 2006). Participants were asked to describe their day-to-day role within the DHHS, whether it had changed since they participated in the development programme, or whether their feelings about it had changed. They were also asked about the workplace activities and the academic programme that followed the multi-day course. This process of reflection, rephrasing and probing allowed for unanticipated responses and issues to emerge (Tod, 2006), with the focus on letting the story, as ‘owned’ by the interviewee, to surface. Each interview took at least an hour, was recorded, transcribed by the graduate trainee and returned to the participant for quality assurance. Only when the transcription was returned with approval was it added to the data file for further analysis.
Data analysis and reporting
Data were analysed through systematic grounded theory techniques (Charmaz, 2006). The researchers reviewed the emerging themes separately then discussed their ideas. During the analysing process of the qualitative component, notes were kept in the form of memos, which described the researcher’s impressions, ideas and perceptions. The data was coded and sorted, leading to the discovery of themes (Charmaz, 2006). A set of six themes emerged from the data for further exploration and reporting.
Evaluation survey results
Agentic resources (General Self-Efficacy Scale): The item-level statistics for the GSES are listed in Table 6. The mean scores for each item at T1 (prior to attending the development programme course) and T2 (nine months following the development programme course) are listed. These item-level means fall within the range of means recorded in previous cross-national research (Liu & Lee, 2012; Scholz et al., 2002) with the exception of one item: ‘I am confident that I could deal efficiently with unexpected events’ scored higher (M = 3.49) in this study than in previous studies (M = 3.40).
|
TABLE 6: Results for the General Self-Efficacy Scale: Item-level statistics.
|
Strong results (P < 0.01) included an increase in the ability to overcome opposition, an increase in a sense of resourcefulness, coping abilities and problem solving. Moderate results (P < 0.05) included increases in the ability to solve difficult or intractable problems, and the ability to deal with unexpected events or issues. A weak increase (P < 0.1) was evident in the response to finding solutions to troubles. There was no statistical support (p > 0.1) for the statement that it was ‘easy’ to achieve aims and goals. The overall response, however, indicated a moderate rise in general self-efficacy over time.Relational resources (Berlin Social Support Scale, Perceived Instrumental Support): The BSSS results are listed in Table 7. One significant result (P < 0.01) was around the availability of helpful people. Moderate results (P < 0.05) included the availability of reliable people, and assistance when overwhelmed by demands. There was no statistical support (p > 0.1) for an increase in support for dealing with worries. A weak increase (P < 0.1) was evident in the overall response to the scale, indicating that a slight improvement in social support may have occurred over time.
|
TABLE 7: Results for the Berlin Social Support Scale (instrumental perceived available support): Item-level statistics.
|
Affective resources in the workplace: Table 8 illustrates that there was no support for an increase in positive orientation towards the job in the quantitative results.
Interview results
Increasing confidence: The nine participants who consented to further interviews described a sense of increasing confidence following their participation in the development programme course. Reflecting on their experience over this time, they recalled that this sense intensified as they progressed through the work-based activities. Box 1 provides verbatim extracts that illustrate this theme.
Creating a positive environment: The theme of creating a positive environment relates to the construction of an optimistic work milieu in spite of difficult circumstances emerging within the participants’ professional lives. During the course of this research, as a result of diminishing funding, the DHHS workforce changed and many employees on a fixed-term contract left the organisation. In addition, through intensive restructuring initiatives, certain positions became redundant. All participants in the programme in some way were affected by this change. Even within a less-than-positive organisational environment, however, participants were able to feel somewhat optimistic, as shown by comments in Box 2.
|
BOX 2: Creating a positive environment.
|
Finding networks: The development programme gave participants the opportunity to find and connect with other staff within a large organisation. This most often described as finding networks. In the participant satisfaction survey, of the 110 separate comments on the ‘most valuable part’ of the course, ‘networking’ was mentioned 28 separate times, the strongest single theme. This benefit persisted over time as they progressed through the workplace activities of action learning, coaching and shadowing. Establishing networks led to the realisation that the issues faced in one area were, at times, common to other managers. Listening to what other managers and leaders were doing, trying to share what was happening in the bigger world and using that information led to a rich experience for participants. Box 3 substantiates this with direct quotations from the interviews.
Notable people: The term notable people refers to those peers and senior staff with whom participants were engaged during the development programme. Initially this occurred when senior managers presented at the development programme course. Within the participant satisfaction survey, this emerged in a number of ways, in references to the ‘presenters’, ‘presentations’, ‘strategic’ and ‘leader’, and was the second-strongest theme evident in the initial participant responses. Participants found it invaluable that the most senior people in the organisation would give the time to share their stories with them in how they manage and lead.Later on, connection with notable people occurred through work-based activities such as coaching and shadowing. This was one of the enduring themes of the study: a greater sense of ownership of the strategic direction held by senior management and, at the same time, an increase in the ability to understand challenges faced by those ‘in power’. Box 4 shows this impact.
Broadening horizons: The notion of broadening horizons described participants being able to create a view larger than their immediate work group. It involved moving from a day-to-day point of view to acquiring a strategic view in relation to managing the service. Many participants indicated that they had been working in ‘silos’ and that they possessed a limited understanding of the broader DHHS work environment. Their participation in the development programme gave them a chance to look beyond their own unit. The direct quotes in Box 5 lend support to the importance of this theme.
Clarifying career options: The concept of clarifying career options relates to participants having established a sense of direction regarding their career path. This sense of career clarification was something that was evident almost immediately amongst development programme participants. In the post-course satisfaction survey, 98% of the participants agreed that the programme would be of value to their management and leadership practice in their current position: 45% agreed and 53% strongly agreed. At the same time, however, 99% participants believed that the programme would be even more useful in their future career development: 31% agreed and 68% strongly agreed. The increased levels of confidence reported by participants meant that they were looking beyond their current role, rather than becoming more satisfied in it. However, some participants reflected on their career and came to the conclusion that, even if they were looking for something new, they would not like to be in a non-clinical position. Examples of both results are shown in Box 6.
|
BOX 6: Clarifying career options.
|
Triangulating the data
Table 9 provides a broad overview of the results from this study, with strong and moderately significant quantitative results (as indicated by p-values) from the survey questionnaires aligned with the six qualitative themes that emerged from the data through the application of constructivist grounded theory techniques.
The purpose of this research was to determine whether DHHS staff experienced an increase in target outcomes based on the Roberts et al. (2005) ‘three resources’ model. In order to do so, it needed to develop a leadership development programme evaluation that focused on these resources and was suitable for use in the health and human services sector. It sought to extend this evaluation beyond short-term satisfaction measures to capture outcomes over time. The results of this study demonstrated a partial increase in target outcomes amongst participants in the DHHS development programme. In this, it also partially supported the Roberts et al. (2005) model of agentic, relational and positive affective resources. Participants showed an increase in positive agentic resources: confidence and self-efficacy. Self-efficacy has been described as the positive organisational concept with the strongest claim against the criteria of ‘valid measurement, ease of development and potential management for performance development’ (Luthans, 2002b). Whilst Bandura (1997) differentiates the terms ‘self-efficacy’ and ‘confidence’, the participants in the development programme used the latter word in the sense of being confident in their ability to act to produce successful outcomes within the context of their workplace environment. Participation in the development programme also increased the sense of operating in a more positive organisational environment, in spite of the impact of staff downsizing. The manner in which participants related their sense of increased confidence to their organisational environment supports the idea of leadership learning as occurring through a process of participation and reflection, that builds on knowledge and experience, and occurs within context (Kempster, 2009a). The GSES was effective in measuring the change in positive agentic resources before and after participation in the DHHS leadership development programme. The qualitative component of the study was valuable in articulating how the participants themselves understood the impact of the development programme. These shared reflections provide insight into their process of leadership learning and how this related to their role within the DHHS. The outcome was an increase their resources for leadership, as being confident is an attribute of an effective leader and manager (Kouzes & Posner, 2007). The evaluation also found an increase in positive relational resources: finding networks, having contact with notable people and broadening horizons. Here the quantitative data was less emphatic than the qualitative, which demonstrated that some of the most powerful and enduring outcomes of the DHHS development programme were the supportive relationships formed between participants. The qualitative data also showed the importance of interaction with more senior staff, described as ‘notable people’ (Kempster, 2009b) to inspire leadership learning. A stronger sense of horizontal and vertical connectedness within the organisation had the effect of ‘broadening the horizons’ of participants as they gained a greater understanding of their role in relation to others. It may be that the BSSS was not the optimum measure for this set of resources, in this context, and further research is required in order to determine if a suitable replacement may be found amongst the many social support surveys available (Hietzmann & Kaplan, 1988). The qualitative data also emphasised the importance of the workplace activities associated with the development programme in providing opportunities for growing these supportive relationships. The workplace activities were intended to promote observational learning through shadowing (Kempster, 2009b), one-to-one interactive learning through coaching (Brockbank & McGill, 2006) and group learning through action learning (Rigg, 2006). The social and emotional support that participants reported as resulting from these experiences, however, provided relational resources upon which they could draw, sometimes well beyond the timeframe of the development programme itself. With regard to affective resources, the quantitative data showed no change in the level of positive orientation towards the job in this study. At the same time, qualitative data indicated that there was a clarification of career options. Whilst there was no noticeable increase in job satisfaction, there was considerable reflection on whether the particular job was the best ‘fit’ for the individual. These results require more investigation. On the one hand, the results may reflect the stress of working in a ‘downsizing environment’ (Noer, 2009); on the other hand, there may be a deficit in the capture of job affect by the survey instrument. There is some debate about the ability of single items to capture job satisfaction (European Foundation for the Improvement of Living and Working Conditions, 2007; Wanous, Reichers & Hudy, 1997). This may be tested by a comparison of the following year results with this first year, as those results become available. The mixed-method research approach added to the explanatory power of the evaluation through triangulation of the quantitative and qualitative measures. Whilst the GSES measured an increase in self-efficacy, interview data provided insight into the participants’ understanding of how this had occurred and what it meant in their workplace environment. Interviews also captured the strength of the perceived increase in relational resources in a way the BSSS did not. Where the survey question on affective resources did not register a significant result the qualitative data suggested explanations for this outcome. The mixed-method approach used in this study supported the assertion that different types of inquiry are sensitive to different real-world nuances (Patton, 2002). However, the ability of the evaluation to undertake qualitative interviews into the future will depend on resources becoming available. The ability to continue a mixed-method approach is limited and may be best treated as an addition to the evaluation that may be called upon if further clarification of results is required.
Limitations of the study
Both the quantitative and qualitative methods used relied upon the perceptions of the development programme participants, rather than a variety of data sources such as participants’ staff, supervisors or clients and patients. This method was chosen as it enabled participants to contribute to the evaluation with minimal effort, possibly increasing response rates. It also allowed for a potential bias in the results. Whilst this could be partially controlled for in future studies, without significantly altering the methodology, by the inclusion of two questions from the Crowne-Marlowe social desirability scale (Donaldson & Grant-Vallone, 2002), it represents a limitation in the current study. In addition, whilst the survey questionnaires were all administered at a consistent interval from the time of participation in the development programme course, the follow-up qualitative interviews were undertaken at varying time periods, between one and two years after participants had engaged with the programme. This limited their explanatory strength. Finally, the selection of subjects for interviewing was somewhat limited as only 26% of those approached consented to participate.Studies show that the link between self-efficacy and social support can be mutually reinforcing, with social support enabling self-efficacy and self-efficacy cultivating social support, depending on the context (Schwartzer & Knoll, 2007). Until more advanced statistical techniques are applied, these interactions will go unexplored, within the context of the DHHS development programme evaluation.
Conclusion
The results of the development programme evaluation indicated that DHHS staff experienced a moderate increase in self-efficacy, a weak increase in social support and no change in job satisfaction. Participant self-confidence increased and career options became clearer, through contact with wider networks that included senior colleagues and other influential ‘notable people’ to broaden understanding (horizons) and create a more positive organisational environment. The mixed-method approach assisted in the development of an evaluation suitable to the context of the development programme. The quantitative research provided support for use of the GSES in capturing some of these changes and, more broadly, for the theoretical framework developed by Roberts et al. (2005). The BSSS was less effective as a measure for this set of resources, in this context, and further research is required. A need for further research into quantitative measures that may capture increase in positive affective resources has also been flagged. The qualitative research provided support for the position that leadership development is a vital component in meeting the challenges facing the health system (Health Workforce Australia, 2012) as participants drew upon peer networks and senior leadership for practical and inspirational support in the delivery of services to the Tasmanian public. The benefits of workplace learning are not only found at individual level, however, as the development of peer-led and other ‘in-house’ activities have been recognised as a cost-effective approach for leadership development within health and human services (McPherson, 2010).
The authors would like to thank Mick Bendor and Heather Stokes for research assistance.
Competing interest
The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.
Authors’ contributions
E.A.S. (University of Tasmania) was the project leader on this work, responsible for research design, the quantitative survey, literature review and analysis. P.V.D. (University of Tasmania) provided qualitative survey design and analysis. Both authors contributed to writing the article.
Australian Institute of Health and Welfare. (2012). Health expenditure Australia 2010–11. Health and welfare expenditure series. (Vol. 47, cat. no. HWE 56). Canberra, ACT, Australia.Avolio, B.J., Luthans, F., & Youssef, C.M. (2007). Psychological capital: Developing the human competitive edge. New York, NY: Oxford University Press. Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: Freeman. Black, A.M., & Earnest, G.W. (2009). Measuring the outcomes of leadership development programs. Journal of Leadership and Organizational Studies, 16, 184–196.
http://dx.doi.org/10.1177/1548051809339193 Bolden, R. (2011). Distributed leadership in organizations: A review of theory and research. International Journal of Management Reviews, 13, 251–269.
http://dx.doi.org/10.1111/j.1468-2370.2011.00306.x Braithwaite, J., Skinner, C.A., & Doery, M.L. (2011). A values-based health system. Medical Journal of Australia, 194(5), 259–262. Brockbank, A., & McGill, I. (2006). Facilitating reflective learning through mentoring and coaching. London, UK: Kogan Page. Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. Thousand Oaks, CA: Sage Publications. Council of Australian Governments. (2011). National health reform agreement. Retrieved April 26, 2013, from
http://www.federalfinancialrelations.gov.au/content/npa/health_reform/national-agreement.pdf Department of Health and Human Services. (2012). Annual report 2011-12. Hobart, TAS, Australia. Development Guild/DDI. (2002). Evaluating outcomes and impacts: A scan of 55 leadership programs. Brookline, MA: W.K. Kellogg Foundation. DiCicco-Bloom, B., & Crabtree, B. (2006). The qualitative research interview. Medical Education, 40(4), 314–321.
http://dx.doi.org/10.1111/j.1365-2929.2006.02418.x,
PMid:16573666 Dickson, G. (2007). The pan-Canadian health leadership capability framework project. Ontario, Canada: Centre for Health Leadership and Research. Donaldson, S.I., & Grant-Vallone, E.J. (2002). Understanding self-report bias in organizational behavior research. Journal of Business and Psychology, 17(2), 245–260.
http://dx.doi.org/10.1023/A:1019637632584 Doyle, L., Brady, M., & Byrne, G. (2009). An overview of mixed methods research. Journal of Research in Nursing, 14, 175–185.
http://dx.doi.org/10.1177/1744987108093962 Edwards, R., Austen, D., & Altpeter, M. (1998). Managing effectively in an environment of competing values. In R. Edwards, J. Yankey, & M. Altpeter (Eds.), Skills for Effective Management of Nonprofit Organizations (pp. 5–21). Washington, DC: National Association of Social Workers.
PMid:9934038 Emmerik, I.J.H.V., & Euwema, M.C. (2008). The aftermath of organizational restructuring: Destruction of old and development of new social capital. Journal of Managerial Psychology, 23(7), 833–849.
http://dx.doi.org/10.1108/02683940810896367 European Expert Group on Health in Restructuring. (2009). Health in restructuring: Innovative approaches and policy recommendations. Bremen, Germany: University of Bremen. European Foundation for the Improvement of Living and Working Conditions. (2007). Measuring job satisfaction in surveys - Comparative analytical report. Dublin, Ireland: Eurofound. French, S., & Holden, T. (2012). Positive organizational behavior: A buffer for bad news. Business Communication Quarterly, 75(2), 208–220.
http://dx.doi.org/10.1177/1080569912441823 Grove, J.T., Kibel, B.M., & Haas, T. (2005). EvaluLEAD: A guide for shaping and evaluating leadership development programs. Oakland, CA: The Public Health Institute. Hancock, H., & Campbell, S. (2006). Impact of the leading an empowered organisation program. Nursing Standard, 20(19), 41–48.
http://dx.doi.org/10.7748/ns2006.01.20.19.41.c4043 Hannum, K., Martineau, J., & Reinelt, C. (Eds.). (2007). The handbook of leadership development evaluation. San Francisco, CA: Jossey-Bass. Health Workforce Australia. (2012). Leadership for the sustainability of the health system: Part 1 - a literature review. Adelaide, Australia: Health Workforce Australia. Health Workforce Australia. (2013). Australian health leadership framework. Retrieved August 14, 2013, from
http://www.hwa.gov.au/sites/uploads/Health-LEADS-Australia-A4-FINAL.pdf Hietzmann, C.A., & Kaplan, R.M. (1988). Assessment of methods for measuring social support. Health Psychology, 7(1), 75–109.
http://dx.doi.org/10.1037/0278-6133.7.1.75 Jick, T.D. (1979). Mixing qualitative and quantitative methods: Triangulation in action. Administrative Science Quarterly, 24(4), 602–611.
http://dx.doi.org/10.2307/2392366 Johnson, R., Onwuegbuzie, A., & Turner, L. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112–133.
http://dx.doi.org/10.1177/1558689806298224 Kapikiran, S., & Kapikiran, N. (2010). The validity and reliability of the Perceived Available Support Scale. Ankara University, Journal of Faculty of Educational Sciences, 43(2), 51–73. Kempster, S. (2009a). How managers have learnt to lead: Exploring the development of leadership practice. Basingstoke, UK: Palgrave MacMillan.
http://dx.doi.org/10.1057/9780230234741, PMCid:PMC2798646 Kempster, S. (2009b). Observing the invisible. Journal of Management Development, 28(5), 439–456.
http://dx.doi.org/10.1108/02621710910955976 Kosimbei, G., Hanson, K., & English, M. (2011). Do clinical guidelines reduce clinician dependent costs? Health Research Policy and Systems, 9(24). Available from
http://www.health-policy-systems.com/content/9/1/24,
PMid:21679458 Kouzes, J.M., & Posner, B.Z. (2007). The leadership challenge. (4th edn.). San Francisco, CA: Jossey Bass. Kvale, S. (1996). Interviews: An introduction to qualitative research interviewing. Thousand Oaks, CA: Sage Publications. Lee, G., & Teo, A. (2005). Organizational restructuring: Impact on trust and work satisfaction. Asia Pacific Journal of Management, 22, 23–29.
http://dx.doi.org/10.1007/s10490-005-6416-6 Leech, N., & Onwuegbuzie, A. (2009). A typology of mixed methods research designs. Qual Quant, 43, 265–275.
http://dx.doi.org/10.1007/s11135-007-9105-3 Liu, W.K., & Lee, Y.-s. (2012). Assessment of cultural dimensions, leadership behaviors and leadership self-efficacy: Examination of multinational corporations in Taiwan. International Proceedings of Economics Development and Research, 28, 199–204. Luszczynska, A., Scholz, U., & Schwarzer, R. (2005). The General Self-Efficacy Scale: Multicultural validation studies. The Journal of Psychology, 139(5), 439–457.
http://dx.doi.org/10.3200/JRLP.139.5.439-457 Luthans, F. (2002a). The need for and meaning of positive organizational behavior. Journal of Organizational Behavior, 23, 695–706.
http://dx.doi.org/10.1002/job.165 Luthans, F. (2002b). Positive organizational behavior: Developing and managing psychological strengths. The Academy of Management Executive, 16(1), 57–72. Luthans, F., Youssef, C.M., & Avolio, B.J. (2007a). Psychological capital. New York, NY: Oxford University Press. Luthans, F., Youssef, C.M., & Avolio, B.J. (2007b). Psychological capital: Investing and developing positive organizational behavior. In D. Nelson, & C.P. Cooper (Eds.), Positive organizational behavior (pp. 9–24). London, UK: Sage Publications.
http://dx.doi.org/10.4135/9781446212752.n2 McLean, S., & Moss, G. (2003). They’re happy, but did they make a difference? Applying Kirkpatrick’s framework to the evaluation of a national leadership program. The Canadian Journal of Program Evaluation, 18(1), 1–23. McPherson, B. (2010). Low-cost high-quality leadership development: A model for an affordable management-development programme. Training and Management Development Methods, 24, 01.01–01.05. NHS Institute for Innovation and Improvement. (2010). NHS leadership framework: A summary. Retrieved April 11, 2013, from
http://www.leadershipacademy.nhs.uk/discover/leadership-framework/ Noer, D.M. (2009). Healing the wounds: Overcoming the trauma of layoffs and revitalizing downsized organizations. San Francisco, CA: Jossey-Bass. O’Byrne, M. (2011). Health budget savings. Tasmanian Government Media Release. Hobart, TAS, Australia: Tasmanian State Government. O’Byrne, M. (2012). Keeping health strong. Tasmanian Government Media Release. Hobart, TAS, Australia: Tasmanian State Government. Onwuegbuzie, A.J., & Leech, N.L. (2005). On becoming a pragmatic researcher: The importance of combining quantitative and qualitative research methodologies. International Journal of Social Research Methodology, 8(5), 375–387. Organisation for Economic Co-operation and Development (OECD). (2011). Health at a glance 2011: OECD indicators. Retrieved April 11, 2013.
http://dx.doi.org/10.1787/health_glance-2011-en Patton, M.Q. (2002). Qualitative research and evaluation methods. Thousand Oaks, CA: Sage Publications. Quinn, R.E. (1988). Beyond rational management: Mastering the paradox and competing demands of high performance. San Francisco, CA: Jossey-Bass. Quinn, R.E. (2005). Moments of greatness: Entering the fundamental state of leadership. Harvard Business Review, 83(7/8), 74–83. Rigg, C. (2006). Understanding the organizational potential of action learning. In C. Rigg, & S. Richards (Eds.), Action learning, leadership and organizational development in public services (pp. 41–51). New York, NY: Routledge. Roberts, L.M., Dutton, J.E., Spreitzer, G.M., Heaphy, E.D., & Quinn, R.E. (2005). Composing the reflected best-self portrait: Building pathways for becoming extraordinary in work organizations Academy of Management Review 30(4), 712–736.
http://dx.doi.org/10.5465/AMR.2005.18378874 Russon, C., & Reinelt, C. (2004). The results of an evaluation scan of 55 leadership development programs. Journal of Leadership and Organizational Studies, 10(3), 104–107.
http://dx.doi.org/10.1177/107179190401000309 Scholz, U., Dona, B. G., Sud, S., & Schwarzer, R. (2002). Is general self-efficacy a universal construct? European Journal of Psychological Assessment, 18(3), 242–251.
http://dx.doi.org/10.1027//1015-5759.18.3.242 Schulz, U., & Schwarzer, R. (2003). Social support and coping with illness: The Berlin Social Support Scales. Diagnostica, 49, 73–82.
http://dx.doi.org/10.1026//0012-1924.49.2.73 Schwarzer, R., Bäßler, J., Kwiatek, P., Schröder, K., & Zhang, J.X. (1997). The assessment of optimistic self-beliefs: Comparison of the German, Spanish, and Chinese versions of the General Self-Efficacy Scale. Applied Psychology, 46(1), 69–88.
http://dx.doi.org/10.1111/j.1464-0597.1997.tb01096.x Schwarzer, R., & Jerusalem, M. (1995). Generalized self efficacy scale. In J. Weinman, S. Wright, & M. Johnston (Eds.), Measures in health psychology: A user’s portfolio (pp. 35–37). Windsor, UK: NFER-Nelson. Schwartzer, R., & Knoll, N. (2007). Functional roles of social support within the stress and coping process: A theoretical and empirical overview. International Journal of Psychology, 42(4), 243–252.
http://dx.doi.org/10.1080/00207590701396641 Schwarzer, R., & Leppin, A. (1988). Social support: the many faces of helpful social interactions. International Journal of Educational Research, 12(3), 333–345.
http://dx.doi.org/10.1016/0883-0355(88)90010-9 Shannon, E., & Burchill, T.A. (2013). ‘Shaping our workforce’: A Tasmanian development program. Australian Health Review, 37(1), 131–133.
http://dx.doi.org/10.1071/AH12142,
PMid:23369870 Shannon, E., Holden, J., & Van Dam, P. (2012). Implementing national health reform - is organisational culture the key? In Conference Proceedings: Australian Political Studies Association Conference (pp. 883–896). Hobart, TAS, Australia: Australian Political Studies Association Conference. Shannon, E., Van Dam, P., & Stokes, H. (2012). Building resilience in a professional services community: The role of leadership development. In Conference Proceedings: 52nd Annual Adult Learning Australia Conference: Lifelong learning = resilient communities (pp. 37–44). Byron Bay, NSW, Australia: Adult Learning Australia Inc. Swanson, V., & Power, K. (2001). Employees’ perceptions of organizational restructuring: The role of social support. Work and Stress, 15(2), 161–178.
http://dx.doi.org/10.1080/02678370110066995 Tod, A. (2006). Interviewing. In K. Gerrish, & A. Lacey (Eds.), The research process in nursing (pp. 337–352). Oxford, UK: Blackwell Publishing. Van Beek, A., & Gerritsen, D. (2010). The relationship between organizational culture of nursing staff and quality of care for residents with dementia: Questionnaire surveys and systematic observations in nursing homes. International Journal of Nursing Studies, 47(10), 1274–1282.
http://dx.doi.org/10.1016/j.ijnurstu.2010.02.010,
PMid:20371058 Wanous, J.P., Reichers, A.E., & Hudy, M.J. (1997). Overall job satisfaction: How good are single-item measures? Applied Psychology, 82(2), 247–252.
http://dx.doi.org/10.1037/0021-9010.82.2.247 World Health Organization (WHO). (2010). Health systems financing. The World Health Report. Geneva, Switzerland.
|