Flying blind, or going with the flow?: Using constructivist evaluation to manage the unexpected in the GraniteNet project

Catherine Arden ¹, Kathryn McLachlan², Trevor Cooper³

University of Southern Queensland. Email: Catherine.Arden@usq.edu.au
Community Development Services Inc, Stanthorpe
Stanthorpe

INTRODUCTION

The GraniteNet Project is a research and development collaboration between the University of Southern Queensland, Australia, and the community of Stanthorpe - a rural community of just over 10,000 people located approximately 240 km from the university campus on Queensland's southern border with New South Wales. The vision of this Community Informatics project, which commenced in 2006 and is now in its fourth phase, is the development of a sustainable community designed, owned and managed web portal that will support Stanthorpe's development as a learning community.

Typical of smaller, rural communities located to the west of the Great Dividing Range that runs the length of Australia's east coast, separating the more heavily populated metropolitan centres from dispersed, inland towns, Stanthorpe has an ageing community, a low median income, a lower proportion of the population with post-compulsory education qualifications and lower use of information communication technologies (ICTs) in comparison with Brisbane metropolitan and larger coastal centres in Queensland (ABS, 2001, 2006, cited in Cavaye, 2008). These are all considered risk factors in terms of the community's continued prosperity and longer term sustainability (Arden, McLachlan & Cooper, 2008). Perceived benefits of GraniteNet for Stanthorpe include that it will be a tool that people of all ages and from all sectors of the community can use to share information, promote community activities and events, and promote and foster learning opportunities. It is hoped that GraniteNet will become a valuable community asset that will enhance existing social networks, provide opportunities for growth and development and help to bridge the 'digital divide' that is said to exist between rural and metropolitan communities (OECD, 2000; FutureLab, 2007).

The project has been moving through a series of developmental phases underpinned by principles of participatory design, community development and capacity-building through Participatory Action Research (PAR) and "cogenerative" learning ¹ (see Elden & Levin, 1991 as cited in Arden, McLachlan & Cooper, 2008). The project's phased approach to development and implementation is similar to approaches adopted in many Community Informatics projects around the world and is pragmatic in terms of its alignment with iterative participatory action research and evaluation cycles, experimental and participatory design processes and the discontinuous nature of project funding. The project phases and timelines are shown in Table 1 below.

Table 1: GraniteNet Project Phases and Timelines²

Phase 1

2006-7

(1^st PAR cycle)

Development of project concept and “business case” for portal

Evaluation of university-community partnership

Phase 2

2007-8

(2^nd PAR cycle)

Design, development and evaluation of pilot “GraniteNet” community portal environment
Review of learning community progress
Phase 2 evaluation

Phase 3

2009

(3^rd PAR cycle)

Consolidation and focused action to build capacity to address sustainability factory
Collaborative, critical review of GraniteNet as a community learning project

With grant monies received from the Queensland State Government targeted to support rural community development initiatives, the GraniteNet Board commissioned an evaluation of the second phase of the project which focussed on the design, development and trial of an incubator community portal environment, a portal governance framework and community engagement strategy. Evaluation aims included documenting the project, establishing an evidence base to inform future decision-making, identifying and exploring significant contextual factors impacting on the project, evaluating the effectiveness of the models and processes used to guide the project, and building a culture of evaluation that would help to ensure ongoing review and critical reflection on progress. Another aim was to add to the existing body of knowledge about the benefits of PAR and evaluation collaborations for community learning and more effective university-community engagement, and the extent to which these methodologies support the achievement of community-identified goals. By documenting and evaluating the design and implementation of GraniteNet, the project team - working in collaboration with the University of Southern Queensland - hoped to be able to make a contribution to knowledge through the identification of critical success and key sustainability factors that might prove to be transferable to other community contexts.

This paper reports the evaluation processes and outcomes, with a focus on exploring the ways in which these methodologies can be used to help Community Informatics researchers and practitioners learn from and about the unexpected. In so doing, the authors explore the nature of the unexpected and unanticipated in the context of GraniteNet and propose that PAR and constructivist evaluation methodologies provide a useful mechanism for dealing with the unexpected in Community Informatics projects such as GraniteNet.

LITERATURE REVIEW

Learning is at the centre of the GraniteNet project, and a shared valuing of and commitment to ongoing learning permeates the project by virtue of the fact that GraniteNet was originally conceived of as a strategy to support the development of Stanthorpe as a "learning community". The emergence of the learning communities movement in the UK, Europe, Canada and Australia during the last three decades of the 20th century as a strategy for supporting development and capacity-building of towns, cities and regions through a "learning based approach to community development" (see Faris, 2005 p. 31; NIACE, 2005; Longworth, 2006) included the promotion of ICTs a strategy for supporting knowledge transfer and innovation through community networking, enhanced opportunities for participation in civil society (e-democracy) and participation in formal and informal learning (see, for example, Coleman & Goetze, 2001; Longworth, 2006). More recently, the rapid and widespread social adoption of ICT innovations has seen the emergence across the world of an increasing number of community technology projects (the study of which is the business of the interdisciplinary field of Community Informatics) which aim to use ICTs to enable communities (Gurstein, 2000). As one such learning community and Community Informatics initiative, the GraniteNet project aims to maximise the use of ICTs to support individual development and community capacity-building ³ through learning.

The important relationship between access to and effective use of ICTs, individual capability⁴, community capacity⁵ and regional development is widely acknowledged in the literature, with particular reference to the existence of a so-called digital divide between those individuals and communities that have access to and make effective use of ICTs, and those that do not. Although Warschauer has warned against the tendency to conceptualise this digital divide in simplistic terms that overemphasise the technology, preferring to use the term "technology for social inclusion" (2002, pp. 25, 27), there appears to be general agreement that people and communities who lack affordable, accessible and reliable ICT infrastructure as well as the digital literacy to make effective use of the technology for participation in formal and informal learning and civil society are at a significant disadvantage in terms of their economic and scultural development (see for example OECD, 2000; Council of Australian State Libraries, 2004; Sevigny & Prevost, 2006). As stated by Sevigny and Prevost, "the debate is no longer whether communities have the means to invest in information technologies, but rather whether they can afford not to" (2006, p.128). This is an issue of concern to rural and regional communities, and in particular those that are more isolated through the so-called tyranny of distance, which impacts on people's opportunities to access and participate in education, training, and enterprise development to the same degree as their metropolitan counterparts.

As is often the case with well-intentioned community development initiatives, and particularly those with a focus on promoting ICTs as a development tool, there are significant implementation challenges. A review of Community Informatics projects in Australia and overseas highlights sustainability as a key problem faced by project teams, evidenced by the plethora of community websites and information technology projects that, after an initial flurry of activity, slowly lose momentum and relevance and become disused relics (see Loader & Keeble, 2004; Schauder, Stillman & Johanson, 2004; Warschauer, 2002 for specific examples). Interestingly, the town of Stanthorpe has its own experience of this story with the earlier version of Granitenet which was rolled out in 2000 as part of a well-funded national ICT initiative⁶, but ultimately proved unsuccessful due in no small measure to a lack of attention to the "complex array of factors encompassing physical, digital, human, and social resources and relationships" that go to make up "the human and social systems that must also change for technology to make a difference" (Warschauer, 2002, pp. 20, 21). Warschauer's position is supported in much of the literature documenting the ebb and flow of community technology projects, as exemplified in the Journal of Community Informatics special theme issue on Sustainability and Community ICTs (2005). Hearn, Kimber, Lennie and Simpson (2005), Knox (2005), and Simpson (2005), who have all conducted research into Australian, rural CI projects, identify broad community involvement and ownership as critical success factors. In fact, Knox goes as far as to recommend that to maximise sustainability, community technology projects need to involve as many people in the community as possible at all stages of the process - a recommendation likely to strike fear into the heart of the most intrepid community development practitioner!

This is related to another factor that strikes fear into the heart - this time of the CI researcher and practitioner: that is, the complexity of the phenomenon that is Community Informatics, and inherent in this complexity, the nature and impact of the "unexpected" in Community Informatics projects. In their attempts to understand, theorise about and deal with the complexity of social and socio-technical problems and challenges they face in their work, CI researchers and practitioners have drawn on systems thinking, chaos theory, social capital theory, complexity theory, structuration theory, and social planning theory, to name just a few. Although Warschauer acknowledges the complexities of projects such as these that aim to use ICTs for the purposes of community development, recognising that they often run into "unexpected difficulties that hinder results", he maintains that these problems are "neither isolated, nor random", but on the contrary, that the "same types of problems occur again and again", and that they relate specifically to the aforementioned overemphasis on technology at the expense of consideration of the human factors (2002, 20). Other CI researchers are in agreement with this premise, emphasising the need to recognise factors such as community ICT readiness (Hearn et al, 2005), the levels of social capital within a community (Simpson, 2005), market related mechanisms (Gurstein, 2001), the degree of embeddedness in the community, and relevance to community needs and goals (Knox, 2005), along with the capacity of the distributed system or network to adapt to changing needs and circumstances (that is, learn) (Stillman & Stoecker, 2004; Graham, 2005) and even to undergo complete transformation (Schauder, Stillman & Johanson, 2004).

Intent on learning from the experience of the previous, failed community technology project mentioned earlier, and the experiences of others shared in the CI literature, the GraniteNet project team has attempted to maximise the likelihood of success by aiming for high levels of community ownership, participation and engagement and adopting strategies aimed at supporting the development of capacity within the community to learn about and use ICTs effectively. These strategies included: structuring the project as a phased PAR project with a strong focus on community engagement and participatory design approaches such as those recommended by Hearn et al (2005) and reported by Merkel , Xiao, Farooq, Ganoe, Lee, Carroll et al (2004); a flexible, 'mash-up' portal design using open source software to maximise affordability and adaptability; development of an ongoing research and development partnership with the local university; and a focus on learning through formative evaluation (see Arden, 2009; McLachlan & Arden, 2009; Arden, McLachlan & Cooper, 2009). It is this formative evaluation strategy, and how it has been used to help the GraniteNet project team plan for, learn from and manage the unexpected in the interest of sustainability that is the focus of this paper.

METHODOLOGY

Community Informatics initiatives such as GraniteNet which aim to bridge the digital divide need to facilitate both access and empowerment (Loader & Keeble, 2004). It follows that research and evaluation of Community Informatics projects should be guided by methods that serve to empower and build capacity in communities as well as being a tool for monitoring progress towards achievement of stated project objectives. This view is supported by Stillman (2005) and Stillman and Stoecker (2004), who advocate the use of PAR for the research and evaluation of community technology projects for building community capacity that will lead to more effective use of new technologies in the service of community development. The approach taken for the evaluation of Phase 2 of the GraniteNet project therefore drew on principles and processes of PAR and evaluation (Elden & Levin, 1991; Wadsworth, 1997; 1998) in combination with "empowerment" evaluation strategies (Fetterman, 1998; 2002). The evaluation was facilitated by the USQ principal researcher in collaboration with members of the Granite Belt Learners Group, who acted as the Critical Reference Group (CRG) (Wadsworth, 1997) for the evaluation, during the period March to November, 2008. The evaluation framework was adapted from a state government community engagement evaluation strategy and incorporated formative, summative and research evaluation (Queensland Government Department of Communities, 2005). Within this framework, the following evaluation methods - designed to model as well as foster effective community engagement practices - were used to guide evaluation processes:

Establishment of a Critical Reference Group (CRG) of key community stakeholders who collaborated in the planning and design of the project evaluation strategy and data collection instruments, the review and analysis of evaluation data and the identification of recommendations for action
Use of Empowerment Evaluation methods (Fetterman, 1998; 2002) - also known as Fourth Generation and Constructivist Evaluation (Guba & Lincoln, 1989; 2001) - an approach to process evaluation where the evaluator acts as facilitator and coach to support others to conduct evaluation of their own project
Establishment of an online collaborative workspace using the university's learning management system (Moodle), which was designed to serve as a central repository for information, documentation and data and to provide a collaborative working and learning environment for the CRG members and other project participants to supplement face-to-face meetings and workshops.

The three primary components of the project that were subject to evaluation, as illustrated in the evaluation design in Figure 1 on the following page, were:

the governance model adopted for Phase 2 of GraniteNet (Governance Model)
the framework used to guide community engagement activities during Phase 2 (Community Engagement Framework), and
the "incubator" portal environments and strategies trialled during Phase II for the GraniteNet community portal (Portal Environment).

The Critical Reference Group met on five occasions during the period August to November, 2008 to plan, review, analyse and discuss evaluation processes and findings and to identify emerging issues and opportunities for action and improvement in relation to Phase 2 of the GraniteNet project as well as the evaluation process itself. CRG meetings and evaluation workshops were facilitated by the principal researcher making use of empowerment evaluation processes and techniques recommended by Fetterman (2002) (such as revisiting the project mission and vision, taking stock, prioritising components for evaluation, designing strategies and tools for data collection, collaborative rating of performance, critical review of data, and action planning). In addition to participating in these meetings, six CRG members also actively participated in evaluation activities on the Moodle site.

Data sources for the evaluation included 124 community surveys, 10 stakeholder questionnaires, 10 GraniteNet content editor evaluations, 20 interviews with community members with a significant disability, evaluation workshops with key stakeholders, Critical Reference Group meetings and data from the Moodle evaluation site. In total, there were 160 individual respondents to the evaluation. Correlation of data from these sources in combination with analysis of key documents and records, statistical data, activity on the GraniteNet website and a review of relevant literature resulted in a reasonably strong data set for the purposes of answering the evaluation questions. An education session on ethical issues in research and evaluation was conducted with the members of the Critical Reference Group at its first meeting. To date no ethical dilemmas have emerged as a result of this research.

Figure 1: GraniteNet Phase 2 Evaluation Design (Source: Arden, 2009)

EVALUATION FINDINGS

For the purposes of this paper, evaluation findings are summarised in terms of key learnings and critical success factors considered to have particular relevance to the conference theme of learning to manage the unexpected in CI, with a focus on the efficacy of theories, models and frameworks guiding project design. Within the over-arching PAR methodology, each of the three core project activities that were subject to evaluation was guided by a particular model, or framework that had been adopted as a result of recommendations from Phase 1, as shown in Table 2.

Table 2: Theories, models and frameworks guiding GraniteNet project design

Theories Informing Project Methodology (Participatory Action Research Approach)
Social capital theory: university researcher as facilitator and “broker” (Ballatti & Falk, 2001; Kilpatrick, 2000) Participatory Action Research as “cogenerative learning/inquiry” between “insiders” and “outsiders” (Elden & Levin, 1991; Greenwood & Levin, 2005; Wadsworth, 1998)
Community Engagement Framework	Governance Model	“Incubator” Portal Environment
IAP2 Public Participation Spectrum (2004): (Inform – Consult – Involve – Collaborate – Empower) Community engagement plan and objectives	Community 'board' of volunteers auspiced by local community development organization (incorporated association) Sub-committees including Technical Working Party Grant funded project manager role(existing Community Development Worker) Part-time Website Administrator (paid position)	Participatory, scenario-based, prototype design Review of community portals – trials and evaluations of platform and systems Community -developed portal design concept community noticeboard community marketplace my learning space Development of portal critical success factors

The evaluation found that structuring the project as PAR was likely to have contributed to the project staying on track with its objectives through the incorporation of the feedback loop that is an essential part of the action research cycle (similar to a continuous quality improvement cycle) (Arden, 2009). Much more significant that this, however, is the reflexive nature of the PAR process, which provided the mechanism for critical review, questioning of assumptions, and learning through dialogue and experimentation (the latter referred to by project participants as being akin to "flying blind"). At the same time, PAR and evaluation processes, which are seen to facilitate community learning (Shaver & Tudbull, 2002: Kilpatrick, 2000; Ballatti & Falk, 2001; Adams, 2005; Arden, 2007; McLachlan & Arden, 2009), have served to build an evidence base for decision-making and ongoing evaluation of processes and outcomes. The ultimate test, however, is whether the learning is translated into action in subsequent phases of the project.

Governance model

In relation to the governance model adopted for the GraniteNet project (as outlined in Table 2 on the previous page), the evaluation found that project participants and stakeholders saw the volunteer board or committee operating under the auspices of a larger, community-based organisation as a strength because it provided a strong organisational structure and policy framework that allowed for the employment of staff, promoted a professional impression to the community and funding bodies, and represented a community-based, democratic and participatory model that aligned well with the GraniteNet vision, values and philosophy. Deficiencies in the areas of strategic and business planning, management of stakeholder expectations and organisational communication were seen to have impacted negatively on the functioning of the Board and its ability to sustain a critical mass of members in the crucial, early stages of the project. Appointment of a Website Administrator with the requisite skills set, local connections, and passion for the project vision was seen as integral to the success of this phase of the project, and not an easy achievement in terms of the combinations of skills and personal attributes required for the role. Overall, the evaluation found that the governance model was well suited to the developmental stage of the portal, but that greater attention needed to be paid to recruitment of suitably skilled volunteers and paid staff as well as effective leadership, strategic planning and communication processes that provide direction, foster shared vision, provide opportunities for individuals to make a valuable contribution, and ensure that newcomers are appropriately orientated and inducted into the project (Arden, 2009).

Portal Environment

The concept model for the incubator portal environment (shown in Figure 2 below) was developed in consultation with community members at the project start-up workshop at the beginning of Phase 2, with the decision taken to commence development with the "Community Noticeboard" component, followed by the "Community Marketplace", and finally, "My Learning Space".

Figure 2: Portal Concept Diagram developed by participants in the Phase 2 Project Start-up Workshop in March, 2008 (Source: Arden, 2009)

The evaluation found that the decision to build the portal environment from a 'mash-up' of open source (ModX CMS, MediaWiki, WebCalendar, Simple Machines Forum) and freeware (Yahoo!Flickr image gallery, MailChimp newsletter, Survey Monkey and Google Analytics) was sound, and has enabled the GraniteNet site to expand and respond to changing community needs, user requirements and environmental factors. Having said this, the experimental, participatory design process was not without its painful learning experiences that impacted on participant morale and community perceptions, at times threatening to derail the project completely. This was demonstrated by a critical incident relating to the dysfunctional nature of the "Technical Working Party" that was established as part of the project governance model to pull together the "incubator" portal environment. Merely convening a group of community members with a variety of technical skills, experiences and perspectives and expecting them to "magic up" a portal environment based on a rough concept design was naïve in the extreme and demonstrates the importance of adequate scaffolding for such participatory design processes. Fischer, Rohde and Wulf (2009) describe the difficulties experienced by Communities of Interest (COIs) such as these, which they distinguish from Communities of Practice (CoPs) and characterise as being comprised of individuals from highly diverse backgrounds with significantly different levels of knowledge and experience and diverse motivations for participation. Nonetheless, the GraniteNet "Community Noticeboard" feature is functioning well, with approximately 100 community groups listed on the portal, most of whom are designing and managing their own pages as "Content Editors", having participated in the ModX training sessions conducted as part of Phase 2. "Community Marketplace" features have been the focus of Phases 3 and 4 of the project, with a focus on development of business sponsorship and income-generating, fee-for-service activities for local businesses. The concept of "My Learning Space" is still to be formally researched as part of the principal author's doctoral studies.

Community Engagement Framework

The evaluation concluded that the community engagement strategy based on the IAP2 Public Participation Spectrum (2004) had provided a useful framework for conceptualising, organising, planning and evaluating community engagement activities in the project, as illustrated in Figure 3

Figure 3: GraniteNet Community Engagement Continuum (Source: Arden, 2009. Adapted from the IAP2 Public Participation Spectrum, 2004)

Effective management of the delicate dance of community engagement was a significant challenge for the project team. Evaluation data showed that participants and community members became impatient with the slow pace of establishing the portal as something concrete that could be seen by the project team, and more importantly, the broader community (described by respondents as "too much talk and not enough action"). Finding the right balance between process and product - that is, between consultation and engagement required for participatory design and 'ownership', on the one hand, and having something tangible 'up and running' on the other - has possibly been one of the biggest challenges (and, dare I say, "wickedest problems" (see Conklin, 2005)) faced by the core project team. In hindsight, it is possible that an adapted version of the model may need to be developed to accommodate the need for the "show me" factor inherent in Community Informatics projects such as GraniteNet (Garlick & Langworthy, 2004). With the benefit of hindsight, however, it is difficult to see how this could have been done any differently. The findings from the Phase 2 evaluation of the "Empowerment" component of the community engagement framework provided evidence of instrumental, social/communicative and transformative learning occurring for project team members and key stakeholders by virtue of their involvement in the project (Arden, 2009; McLachlan & Arden, 2009).

CONCLUSION

There is no doubt that the evaluation was a learning process for participants (see Arden, 2009; McLachlan & Arden, 2009), although the nature and extent of the learning outcomes were not fully explored as part of the evaluation, which, with the benefit of hindsight, could be considered as a design fault. A systematic approach through the PAR and evaluation cycle helped to ensure that the evaluation engendered critical reflection and critical questioning of assumptions. The review and evaluation conducted by Critical Reference Group allowed for the project team to assimilate new information and accommodate it into project planning and implementation, but more importantly, allowed for reflexive learning that supported the respondents to critically reflect on their own processes, questioning assumptions and exploring diverse perspectives. Empowerment and fourth generation evaluation approaches added the dimension of negotiation, which allowed for the unexpected to emerge as a process of discovery while also facilitating deeper learning and transformation for participants. In so doing, this form of evaluation has the potential to bring to light "what we don't know we don't know" (the "unknown unknowns" recently made famous by Donald Rumsfeld) and to uncover blind spots. As Senge, Scharmer, Jaworski, and Flowers (2004) stated, such blind spots "concern[s] not the what and how - not what leaders do and how they do it - but the who, who we are and the inner place or source from which we operate, both individually and collectively" (p. 15). The question still remains, however, as to how a culture of learning and evaluation can be developed and sustained in this CI project, over time, within a dynamic and changing context and with project participants coming and going? Again, with the benefit of hindsight, the gulf between the intention of building a learning and evaluation culture through formative, empowerment evaluation processes, and the reality of the changing levels of participation, contexts and dynamics that characterise voluntary, community-based projects such as GraniteNet is apparent. Perhaps greater attention to the development of organisational policies and procedures that institutionalise practices that reflect and embed values such as participatory design, learning through reflection and critical questioning, in addition to focusing on learning for individual participants, would serve to more effectively build a culture of learning and evaluation over time in this context.

Notwithstanding the lessons to be learned from all this research and evaluation, it would appear that the challenges facing CI practitioners and researchers resemble the "wicked problems" described by Conklin that tend to occur in projects involving creativity, design and "technical complexity" in a context of "social complexity" (2005, pp. 7, 13, 18, 20). According to Conklin, who is building on the work of the urban planner and designer, Horst Rittel (1972), these factors act as "centrifugal fragmenting forces pulling a project apart", the solution to which is the creation of "a shared understanding about the problem, and shared commitment to the possible solutions" through processes of "rational dialogue among a set of diverse stakeholders" (2005, pp. 8, 17, 21). "Over time", writes Conklin, "one acquires wisdom and experience about the approach to wicked problems, but one is always a beginner in the specifics of a new wicked problem" (2005, p. 8). In this sense, a formative, constructivist evaluation that looks at what is working, what isn't, why/why not - and that questions the assumptions that inform and underpin actions and decisions - is perhaps more useful than asking (summatively) what worked, what didn't, why/why not? The authors propose that the formative evaluation process undertaken for the evaluation of Phase 2 of the GraniteNet project is akin to the rational dialogue among stakeholders described by Conklin, and that participants can indeed, over time, acquire wisdom and experience in dealing effectively with, and utilising, the unexpected to work with unique and emerging problems in dynamic and complex socio-technical systems and complex social environments. In other words, in relation to learning from the unexpected, learning to do the process effectively - to "go with the flow" - is just as important as learning 'lessons' from the outcomes of a summative evaluation. Why? Because it is our ability to do the process effectively that enables us to learn from and manage the unexpected, not merely the lessons we have learned about solving particular problems that may have occurred in the past. Perhaps, the evaluative processes need to have the capacity to focus energies on achieving a common purpose whilst allowing and even encouraging levels of disharmony and dissonance?

Having weathered the storms of Phase 2 of the GraniteNet project, the project team find ourselves now, one year down the track since the completion of the evaluation, and five years on since the commencement of the project, in a position where we are able to look back on what has been achieved, with the realisation that the above models and frameworks that were chosen to guide the design and development of GraniteNet have in fact stood us in good stead, as far as it is possible to tell. To quote Graham (2005), we have been the authors of our own fate; we have expressed our story and our story has expressed us; and "the best stories are the ones that survive" (p. 8). The danger is now that without that element of chaotic energy, order will reign over chaos and GraniteNet will become a cliché and atrophy. Perhaps it is the function of the project team to bring together those disparate interests and, paradoxically, seek to ensure that dissonance and chaos are maintained in the interests of survival. The ongoing 'engagement' challenge remains, however: how to reach the more 'disengaged' and marginalised members of the community so that the supposed benefits to be gained from community technology projects - in terms of bridging the digital divide - are actually realised.

REFERENCES

Adams, D. (2005). A short theory of community capital for public policy. Contributed to PASCAL International Conference: Making Knowledge Work - Building Sustainable Communities through Partnerships in Place Management, Social Capital and Lifelong Learning, hosted by the University of Stirling, Scotland, October 25-28

Arden, C. H. (2007). Reflections on the 'Wider Benefits' of Participatory Action Research: GraniteNet Phoenix Phase I. Contributed to (Re)Presenting Community Virtual Conference 8 October - 9 November 2007, hosted by the University of Southern Queensland, Toowoomba in association with Observatory PASCAL, University of Stirling, UK and RMIT, Victoria.

Arden, C.H. (2009). GraniteNet Phase 2 Evaluation Report. Centre for Research in Transformative Pedagogies, University of Southern Queensland, Toowoomba, Australia. Available from http://granitenet.com.au/about/project

Arden, C. H., McLachlan, K., & Cooper, T. (2008, October 26-28). Building capacity through sustainable engagement: More lessons for the learning community from the GraniteNet project. Contributed to the Australian Learning Communities Network Conference, Hyatt Regency North Terrace, Adelaide, SA.

Arden, C.H., McLachlan, K., & Cooper, T. (2009). Building capacity through sustainable engagement: More lessons for the learning community from the GraniteNet project. Australian Journal of Adult Learning, 4(1), April

Ballatti, J., & Falk, I. (2001), 'Socio-economic contributions of adult learning to community: A social capital perspective'. Paper presented to the Conference on the Wider Benefits of Learning: Understanding and monitoring the consequences of adult learning, European Society for Research on the Education of Adults (ESREA), 13-16 September 2001, Lisbon, Portugal

Cavaye, J. (2008). Stanthorpe 2020: Your say our future community plan. Toowoomba: Cavaye Community Development

Coleman S., & Goetze, J. (2001). Bowling Together: Online Public Engagement in Policy Deliberation. London: The Hansard Society. Available from www.bowlingtogether.net/

Conklin, J. (2005). Wicked problems and social complexity. Ch.1 in Dialogue Mapping: Building Shared Understanding of Wicked Problems. West Sussex: John Wiley & Sons Ltd.

Council of Australian State Libraries. (2004). Submission to Senate Employment, Education and Workplace Relations Committee Enquiry into Future Directions for Lifelong Learning. Available from http://www.nsla.org.au/publications/submissions/2004/pdf/NSLA.Submission-20040617-Progress.And.Direction.Of.Lifelong.Learning.pdf

Elden, M., & Levin, M. (1991). Cogenerative learning: Bringing participation into action research. In W. Foote Whyte (Ed.), Participatory Action Research (pp. 127-42). London: Sage

Faris, R. (2005). Lifelong learning, social capital and place management: A Canadian perspective. In C. Duke, M. Osborne, & B. Wilson (Eds.), Rebalancing the social and economic: Learning partnership and place (pp. 16-36). Leicester: NIACE

Fetterman, D. (1998). Empowerment evaluation and the internet: A synergistic relationship. Current Issues in Education 1(4). Available from http://cie.ed.asu.edu/volume1/number4/

Fetterman, D. (2002). Empowerment evaluation: Building communities of practice and a culture of learning. American Journal of Community Psychology 30(1), (89-102)

Fischer, G., Rohde, M., & Wulf, V. (2009). Community-based learning: the core competency of residential, research-based universities. In J. Carroll (Ed.). Learning in Communities: Interdisciplinary Perspectives on Human Centred Information Technology. (pp. 75-110). London: Springer

Futurelab. (2007). Beyond the digital divide: Rethinking inclusion for the 21st century. Available from http://www.futurelab.org.uk/resources/documents/

Garlick, S., & Langworthy, A. (2004). Building a culture of improvement through evaluation in university/regional community engagement. Swinburne University of Technology. Available from http://www.lilydale.swinburne.edu.au/crd/publications.htm

Graham, G. (2005). Community networking as radical practice. The Journal of Community Informatics 1(3). Available from http://ci-journal.net

Greenwood, D., & Levin, M. (2005). Reform of the social sciences, and of universities through action research. In N. Denzin & Y. Lincoln (Eds.), The Sage Handbook of qualitative Research (3rd ed.) (pp. 43-64). London: Sage

Guba, E., & Lincoln, Y. (1989). Fourth Generation Evaluation. CA: Sage Publications

Guba, E., & Lincoln, Y. (2001, November). Guidelines and Checklist for Constructivist (aka Fourth Generation) Evaluation. Available from www.wmich.edu/evalctr/checklists

Gurstein, M. (2000). Community informatics: Enabling communities with information communication technologies. Hershley, PA: Ideas Group Publishing

Gurstein, M. (2001). Community Learning, Community Development, and the New Economy. Ottawa: Community Learning Networks Secretariat, Office of Learning Technologies, Govt of Canada.

Gurstein (Ed.). (2005). The Journal of Community Informatics Special issue: Sustainability and Community ICTs 1(2), (102-120). Available from http://ci-journal.net

Hearn, G., Kimber, M., Lennie, J., & Simpson, L. (2005). A way forward: Sustainable ICTs and regional sustainability. The Journal of Community Informatics Special issue: Sustainability and Community ICTs 1(2), (18-31). Available from http://ci-journal.net

International Association for Public Participation (IAP2). (2004). IAP2 Public Participation Spectrum. Available from http://www.iap2.org.au/sitebuilder/resources/knowledge/asset/files/36/iap2spectrum.pdf

Kilpatrick, S. (2000). Community learning and sustainability: Practice and policy. Keynote Address to First Conference on the Future of Australia's Country Towns, hosted by LaTrobe University, Bendigo, Victoria, June 29

Knox, I. (2005). Winning ways: Online regional communities that click. Paper presented to the 2nd Future of Australia's Country Towns Conference, Bendigo, Vic, July 11-13

Loader, B., & Keeble, L. (2004) Challenging the Digital Divide? A Literature Review of Community Informatics Initiatives. York: The Joseph Rowntree Foundation.

Longworth, N. (2006). Learning cities, learning regions, learning communities: Lifelong learning and local government. London: Routledge

McLachlan, K., & Arden, C. H. (2009). Community learning projects: Transforming post-compulsory education provision in rural communities. In R. E. Harreveld, G. R. Danaher & P. A. Danaher (Eds.), Sea changes, tree changes and bush lessons: Post-compulsory education and rural renewal. Theme issue of Rural Society, 19(2), (146-162)

Merkel, C., Xiao, L., Farooq, U., Ganoe, C., Lee, R., Carroll, J., et al. (2004). Participatory design in community computing contexts: Tales from the field. Paper presented at the Proceedings of the Eighth Biennial Participatory Design Conference, Toronto, Canada, July 27 - 31

National Institute for Adult Continuing Education. (2005). Rebalancing the social and economic: Learning, partnership and place. Leicester: Author

OECD. (2000). The emerging policy agenda: Learning to bridge the digital divide. Retrieved 20 October, 2008 from http://www.oecd.org/

Queensland Government Department of Communities. (2005). Engaging Queenslanders: Evaluating Community Engagement. Author

Saito, M. (2003). Amartya Sen's capability approach to education: a critical exploration. Journal of Philosophy of Education 37(1), 17-33

Senge, P., Scharmer, O., Jaworski, J., & Flowers, B. (2004). Presence: Human Purpose and the Field of the Future. Cambridge, MA: Society for Organisational Learning

Sevigny, B., & Prevost, P. (2006). The 'learning community' as a local development strategy. The Journal of Community Informatics, 2 (2) Special Section: Learning in Communities. Retrieved 20 August, 2009 from http://ci-journal.net/index.php/ciej/article/view/266/224

Shaver, S., & Tudbull, J. (2002). Literature review on factors contributing to community capability. Final Report prepared for the Department of Family and Community Services under the Deed of Agreement for the Provision of Social Policy Research Services December 2001, Revised June 2002. Retrieved 2 February, 2009 from http://www.Shaver&Tudbull.unsw.edu.au/reports/ComCap%20Report%20Final.pdf

Simpson, L. (2005). Community informatics and sustainability: Why social capital matters. In M. Gurstein (Ed.). The Journal of Community Informatics Special Issue: Sustainability and Community ICTs. 1(2) (102-120) Available from http://ci-journal.net

Schauder, D., Stillman, L., & Johanson, G. (2004). Sustaining and transforming a community network: The information continuum model and the case of VICNET. Refereed Conference paper for CIRN 2004: Sustainability and Community Technology, Monash Centre, Prato Italy, 29 September-1 October

Stillman, L. (2005). Participatory action research for electronic community networking projects. Community Development 36(1), (77-92).

Stillman, L., & Stoecker, R. (2004). Empowerment for the west: Community information, knowledge management and technologies for community service providers in Melbourne's eastern region: a prospective report. Refereed Conference Paper for Building & Bridging Community Networks: Knowledge, Innovation & Diversity through Communication, hosted by University of Brighton, UK, 31 March - 2 April

Wadsworth, Y. (1997). Everyday evaluation on the run (2nd ed.). Sydney: Allen and Unwin

Wadsworth, Y. (1998). What is participatory action research? Action Research International Paper 2. Available from http://www.scu.edu.au/schools/gcm/ar/ari/p-ywadsworth98.html

Warschauer, M. (2002). Reconceptualizing the digital divide. First Monday 7(7) July. Available from http://firstmonday.org

ENDNOTES

¹ The terms "cogenerative learning" and "cogenerative inquiry" are used in the context of PAR to describe a process whereby community members with local knowledge (insiders) engage in collaborative inquiry with professional researchers (outsiders) in "co-creating 'local theory' that the participants test out by acting on it" (Elden & Levin, 1991, pp. 129-130: Greenwood & Levin, 2005, p. 54).
² Adapted from McLachlan & Arden (2009)
³ Understood here as helping to establish conditions under which "the necessary personal and systemic attributes" required to identify and address community development challenges can develop and "be mobilised into action for the good of the community" (Adams, 2005, pp. 4, 5)
⁴ Defined here in Sen's (1987) terms as "what real opportunities you have regarding the life you may lead" (cited in Saito, 2003, p. 21)
⁵ Defined here as "the personal and social characteristics that can be mobilised into action for the good of the community" (Adams, 2005, p. 4)
⁶ See Hearn, Kimber, Lennie & Simpson (2005), Simpson (2005) and Knox (2005) for analyses of Australian community technology projects funded under this program