Leveraging learning technologies for collaborative writing in an online pharmacotherapy course This article was downloaded by: [University of Minnesota Libraries, Twin Cities], [Amy L. Pittenger] On: 30 April 2012, At: 06:18 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Distance Education Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/cdie20 Leveraging learning technologies for collaborative writing in an online pharmacotherapy course Amy L. Pittenger a & Becky Olson-Kellogg b a University of Minnesota, College of Pharmacy, Minneapolis, MN, USA b Program in Physical Therapy, University of Minnesota, Minneapolis, MN, USA Available online: 30 Apr 2012 To cite this article: Amy L. Pittenger & Becky Olson-Kellogg (2012): Leveraging learning technologies for collaborative writing in an online pharmacotherapy course, Distance Education, 33:1, 61-80 To link to this article: http://dx.doi.org/10.1080/01587919.2012.667960 PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.tandfonline.com/page/terms-and- conditions This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material. http://www.tandfonline.com/loi/cdie20 http://dx.doi.org/10.1080/01587919.2012.667960 http://www.tandfonline.com/page/terms-and-conditions http://www.tandfonline.com/page/terms-and-conditions Leveraging learning technologies for collaborative writing in an online pharmacotherapy course Amy L. Pittengera* and Becky Olson-Kelloggb aUniversity of Minnesota, College of Pharmacy, Minneapolis, MN, USA; bProgram in Physical Therapy, University of Minnesota, Minneapolis, MN, USA (Received 16 July 2011; !nal version received 15 December 2011) The purpose of this project was to evaluate the development and delivery of a hypertext case scenario document to be used as the capstone assessment tool for doctoral-level physical therapy students. The integration of Web-based collaborative tools (PBworks™ and Google Sites™) allowed students in this all-online course to apply their pharmacotherapy knowledge in a physical ther- apy patient scenario, while working with colleagues to determine the best route of patient care. Students developed digital writing skills imperative to a patient-centered, collaborative health-care !eld, and practiced evidence-based patient care. The !ndings demonstrate that the implementation of collaborative digital writing with a hypertext document case scenario assessment as the primary assessment tool in this online pharmacotherapy course delivered to doctoral-level physical therapy students is a feasible and effective educational strategy. Keywords: collaborative digital writing; authentic assessment; health profes- sional; whole-task Introduction Most educational programs state that competence in some !eld is the primary educa- tional objective. However, what does it mean to be competent? Most de!nitions combine the use of knowledge, skills, and attitudes to solve problems within a particular !eld (Baartman, Bastiaens, Kirschner, & van der Vleuten, 2007). Van Mer- riënboer (2007) de!ned competence as the effective combination of declarative, pro- cedural, and conditional knowledge, automated routines for solving familiar problems fast and with few errors, metacognitive knowledge to monitor and regulate task-related activities, and the acknowledgement of expertise by the speci!c profes- sional community. Competence, therefore, is also socially constructed, context spe- ci!c, and durable; a person maintains competence in a !eld, even as tools or working methods change. The metacognitive portion of competence in this de!nition is described as “re"ective expertise” (van Merriënboer, 2007, p. 74); self-awareness is important for achieving competence and maintaining it. In short, competence is an integration of knowledge, skills, and attitudes and also acknowledgment of expertise by one’s peers. *Corresponding author. Email: alp@umn.edu Distance Education Vol. 33, No. 1, May 2012, 61–80 ISSN 0158-7919 print/ISSN 1475-0198 online ! 2012 Open and Distance Learning Association of Australia, Inc. http://dx.doi.org/10.1080/01587919.2012.667960 http://www.tandfonline.com D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 If the primary educational goal of educational programs is to graduate students who can competently address complex problems, how does this in"uence instruc- tional strategies? Merrill (2002) identi!ed !ve !rst principles of instruction: Learning is facilitated when: learners are engaged in solving real-world problems, existing knowledge is activated as a foundation for new knowledge, new knowledge is demonstrated to the learner, new knowledge is applied by the learner, and new knowl- edge is integrated into the learner’s world. (p. 44–45) All programs and educational contexts seek to create learning environments that allow learners to apply the content in a personal and authentic way, stimulating greater interest and expanding learning in ways unanticipated by the instructor. Learning, then, requires authentic problems that are complex and ill-de!ned. Real- world settings present complex and ill-de!ned problems whereby the solution is beyond the abilities of any one learner; collaboration with peers is necessary to cre- ate the best solution. Thus, learners are required to actively engage in problem solv- ing with each other, struggle with the many possible solutions, build on prior knowledge, and incorporate and build new understanding (Bransford, Brown, & Cocking, 2000). Theoretical framework Authentic educational strategies to develop clinical competency It is not possible to include all related content into the formal curriculum. There- fore, students must be able to transfer learning from situations encountered within the formal curriculum to real-life scenarios for which they have had no formal exposure. Many educators argue that whole-task, authentic, collaborative educa- tional experiences are required for meaningful learning, which makes transfer possi- ble (Doering & Veletsianos, 2008; Keller, 2008; Merrill, 2002; van Merriënboer, 2007). This educational position requires a post-structuralism, social constructivist philosophy (Weedon, 1997); the instructor must hold that learning is social and that meaning is constructed by the learner (Wang, 2008). Implementation of whole-task and social interaction as learning strategies requires a social constructivist episte- mology; one has to accept that learning is constructed and that there is not one “T”ruth or one “R”eality that is handed down from teacher to student; this philo- sophical view also holds that meaningful learning does not occur in isolation (Amhag & Jakobsson, 2009; Cole, 2009; Duffy & Cunningham, 1996; Jonassen, 2000; Kramarae, 2007; Paulus, 2005). By incorporating social interaction into the design of the learning environment, the constructivist philosophy becomes that of social constructivism, which differs from cognitive constructivism on the essential component of collaborative learning (Wang, 2008), and incorporates the socially constructed concept of competence. Using learning technologies to provide scaffolding The idea that learning requires grappling with authentic problems with ill-de!ned solutions is also consistent with cognitive complexity theory, which argues that both cognition and affect domains must be engaged (Tennyson & Breuer, 2002), and motivational design theory (Keller, 1987, 2008), which argues that educational 62 A. L. Pittenger and B. Olson-Kellogg D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 problems must have personal relevance to attract and hold learners’ attention. However, ill-de!ned, authentic tasks are often too complicated for developing clinicians. As Merrill (2002) emphasized, students may not have the prior knowl- edge necessary to engage in an authentic, complex task. Merrill also emphasized the importance of supporting students with educational scaffolding. The provision of student support through demonstration, feedback, and necessary resources ensures that students will not become frustrated and give up; this also builds learner con!dence that successful completion is possible and worthwhile (Keller, 1987, 2008). Merrill also argued that learners must have the opportunity to “go public” with knowledge as a means to socially construct competence, through demonstra- tion and defense of knowledge, skills, and attitudes. The argument that students must apply, practice, and struggle with the material in a way that is purposeful, situated, and collaborative has support in many theorists’ works, such as Vygotsky, Dewey, Bruner, Whitehead, Duffy, Jonassen, Lave and Wenger, Wertheimer, and others (as cited in Duffy & Cunningham, 1996; Hull & Saxon, 2009; Jonassen, 2000; Lave & Wenger, 1991; Lund, 2008). Most importantly, these theoretical frameworks support the overall implementation of collaborative, complex learning experiences (coupled with necessary learner scaffolding) that engage learner cogni- tive and affective domains to facilitate meaningful learning and knowledge transfer. Language and writing as strategies for learning Many have argued in favor of the social nature of learning (Gee, 2005; Lave & Wenger, 1991; Lewis & Ketter, 2004). Many, too, have argued for the role of lan- guage and writing in learning (Alverman, O’Brien, & Dillon, 1996; Bean, 2001; Berninger & Winn, 2006; Hull & Saxon, 2009; Sosnoski, 1999). The combining of learning, language, and social interaction has been de!ned as “sociogenesis—that we come to knowledge by taking part in collective activities that evolve over time, and where language and material artifacts function as collective structural resources” (Lund, 2008, p. 36). As students join a profession, they not only adopt the knowledge of the group, but also take on ways of thinking, acting, and commu- nicating; they take on the identity of someone in that profession. This is what Gee (2005) described as “D”iscourse (p. 7) and is also consistent with the socially con- structed de!nition of competence: the assertion that competence is not only a matter of skill, but also acknowledgment from colleagues. If competency is the educational goal, then competency is not only demonstrated through knowledge acquisition, application, and integration, but also demonstrated through “D”iscourse. One way for students to demonstrate that they are thinking, acting, and communicating like a member of a professional !eld is to ask them to collabora- tively create written communications appropriate for different audiences, namely, for the students in this project, patients, and other members of the health-care team. According to a sociocultural view, learning is both a social and collaborative process, where meaning is co-constructed and co-authored (Amhag & Jakobsson, 2009). Language is central to this social construction of meaning. As Weedon (1997) explained, language is “the place where our sense of ourselves, our subjectivity, is constructed” (p. 21). Providing students with a platform to collabora- tively grapple with constructing audience-appropriate written communications requires students to engage with, and at the same time, identify with, the “D”iscourse of the profession. This strategy also requires students to demonstrate Distance Education 63 D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 the application of the course content in a way that traditional, individual exams do not. Learning technologies provide a mechanism through which the necessary scaffolding can be provided, so that learning experiences can be complex, authentic, and collaborative, and thereby motivating and meaningful. Leveraging learning technologies for collaborative writing in an all-online course This redesign project used collaborative writing as a teaching strategy to model effective professional communication and as an assessment component to demon- strate content mastery. Bean (2001) eloquently described the connection between writing and learning: “When we make students struggle with their writing, we are making them struggle with thought itself” (p. xiii). Writing has traditionally been a solitary activity, and one that has been dif!cult to scaffold in a timely manner, yet many have demonstrated the educational affordances of co-writing as a central ped- agogical strategy (Trentin, 2009). Digital writing allows for the incorporation of an authentic communication component into a course and provides timely scaffolding in a logistically manageable way. For example, wikis can be used for creating col- laborative writing and hypertext documents for showcasing !nal communication products (Landow, 2006; Lund, 2008). Combining collaborative learning and writ- ing communication using digital writing is one potential strategy to develop com- petency in health professional students. Writing challenges students to master content and discourse by collaboratively constructing meaning, while practicing being a health-care provider; what Gee (2003) referred to as projected identity. Here was an opportunity for using learning technology to purposely meet an edu- cational need. Use of collaborative learning experiences are also important for health professionals, such as the physical therapy students speci!c to this project, who will deliver care in a collaborative team once in practice; students, then, bene!t from learning to negotiate strategies for effective communications prior to entry into the professional setting. In a review of the literature, Rourke and Kanuka (2009) reported that measures of student learning in online collaborative environments remain an unexplored area of research, and that educators rely on student perception of learning and application of potentially invalid student assessment strategies. They concluded that there does not appear to be consistent, deep, or meaningful learning within online communities of inquiry. In addition to implementing authentic learning experiences, an authentic assessment strategy was used, which many have argued is essential for effective and engaged learning (Jonassen, 2000; Merrill, 2002). In this course redesign, whole-task, competency-based learning (4C/ID instructional design model) utilizing collaborative learning and !rst principles of instruction strategies was integrated within a pharmacotherapy course to doctoral- level physical therapy students. In this instructional design model, the C refers to the four components of learning tasks, supportive information, just-in-time information, and part-task practice. The advantage of this model is that it allows instructors and designers to create learning experiences that are complex and authentic because of the supportive design features of scaffolding, practice opportunities, and procedural information (van Merriënboer, 2007). See Table 1 for additional description of how the four components of the 4C/ID model were de!ned in the course design. 64 A. L. Pittenger and B. Olson-Kellogg D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 Context of the study Phar 6800: Rehabilitation Pharmacotherapy is a doctoral-level course for students enrolled in a physical therapy program at a large Midwestern university, starting the second year of a continuous three-year program. The required pharmacotherapy course for these students has always been offered by the University of Minnesota’s College of Pharmacy in a distance format, traditionally with online lectures, study guides, and practice cases with the !nal grade determined by an average score from four multiple-choice exams. In the redesigned offering, students were divided into small groups of !ve stu- dents at the beginning of the summer semester; typically, 50 students enroll for the course each summer semester, creating 10 groups of 5 students. Groups were assigned unique, complex patient case scenarios (involving approximately 18 medi- cations), containing six major pharmacotherapy issues (cardiovascular, infectious disease, central nervous system/pain, endocrine, gastrointestinal, respiratory), corre- lating with each of the content sections of the course. Please see Figures 1–3 for screen shot examples of a case scenario, medication list, and referring physician and medical chart information. (The images are of actors, not real patients or physi- cians. Medical scenario is !ctitious and any reference or similarity to actual persons is purely coincidental and unintentional.) To ensure the cases were as authentic as possible to the physical therapy practice setting, the cases were collaboratively developed between the College of Pharmacy and the Program in Physical Therapy; two PhD candidates from the physical therapy program, already licensed in physical therapy, wrote the physical evaluation components of the cases, using physical ther- apy speci!c language and format. These candidates also provided detailed feedback to students as they completed the steps of the case assignment and served as the two independent raters for the !nal project. Groups were required to identify the signi!cant pharmacotherapy issues present within the case and develop their responses in a PBworks™ (http://pbworks.com/) course wiki. Please see Table 2 for an example of the table students were required to complete within the wiki and Google Sites (http://www.sites.google.com) environments. Groups were asked to collaboratively create physical therapy treatment plans within a pharmacotherapy context—that is, choose and justify therapy regimens based on the pharmacotherapy risks speci!c to the case scenario—using the ICF (International Classi!cation of Functioning, Disability and Health) model (Steiner et al., 2002). The work-in-progress was accomplished within the wiki site. For their !nal project, groups submitted in Google Sites a hypertext document demonstrating evidence-based recommendations (recommendations were linked to supporting evi- dence within the Google Sites !nal case solution) appropriate for three levels of audience: the patient, referring physician, and physical therapy colleagues. The three levels of audience build from Fairclough’s explanation of how language !gures into Table 1. 4C/ID model and corresponding redesigned course features. 4C/ID model Course feature Whole-task Very complicated patient case Scaffolding Case divided up into steps Online lectures and materials Additional practice opportunities Smaller cases, with solutions provided Procedural information Daily feedback/opportunity to use instructional team Distance Education 65 D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 http://pbworks.com/ http://www.sites.google.com Figure 2. Screen shot of medication list example. Patient scenario is !ctitious and any similarity to an actual person or persons is purely coincidental and unintentional. Figure 1. Screen shot of patient case scenario example. The image is of an actor, not a real patient. Patient scenario is !ctitious and any similarity to an actual person or persons is purely coincidental and unintentional. 66 A. L. Pittenger and B. Olson-Kellogg D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 social practice (2004); namely, “ways of acting, ways of representing, and ways of being” (p. 228). Not only were students using language to act discursively and represent the practice of physical therapy, but also they used language as a way of self-identifying as a physical therapist (Fairclough, 2004; Gee, 2003). Students were required to provide hyperlinks to evidence appropriate for each respective audience within their recommendations. Final collaborative hypertext projects were also made available to faculty within the physical therapy program, providing students with the additional public component of competency development (Merrill, 2002), as well as an opportunity to begin to develop professional “D”iscourses not only within their groups and class, but also within their practicing professional commu- nity (Gee, 2005). Purpose of the study The purpose of this study was to investigate the use of hypertext document case scenario assessment for use as the primary assessment tool in this online pharmaco- therapy course delivered to doctoral-level physical therapy students. Speci!cally, the questions we sought to answer were: Figure 3. Screen shot of referring physician and medical chart information example. The image is of an actor, not a real physician. Medical referral scenario is !ctitious and any similarity to an actual person or persons is purely coincidental and unintentional. Table 2. Column headings of pharmacy table completed by students in part 1 of the course. Condition medication is treating Brand name Generic name Two agents in the same class Basic mechanism of action Common side effects How this treatment impacts PT Distance Education 67 D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 (1) To what extent does collaborative writing within a wiki effectively facilitate learning? (2) Is it feasible to use a completed hypertext document to demonstrate content mastery and health professional competency? (3) How does working within a group, addressing interprofessional audiences as well as a patient audience, impact professional identity development? Methods and materials Data collected This study included what Patton (2002) referred to as a “triangulation of data sources and analytical perspectives to increase the accuracy and credibility of !nd- ing” (p. 93). Multiple sources of quantitative and qualitative data were collected for the purposes of increasing breadth and depth of understanding of the research ques- tions posed (Creswell, 2009, 2010; Patton, 2002). Entrance survey and course evaluation The entrance survey and course evaluation were developed using the guidelines described by Gaddis (1998) and Dillman, Tortora, and Bowker (1998). Following the initial survey development, a health-care professional student took the two sur- veys using the “think aloud” approach (Collins, 2003), where an investigator sat with a student as she took the survey and she described what she thought the sur- vey was asking her, what she was thinking as she responded, and any dif!culties she was having completing the survey. Based on these comments, the survey was revised. The survey was then piloted with !ve students not involved in the study, resulting in further, minor modi!cations of the survey. At the beginning of the semester, students were invited to complete the consent form and survey via a hypertext link sent in an email, which also assured the anonymity of responses, and were sent one reminder noti!cation to complete the survey one week follow- ing the original invitation. The course evaluation was delivered similarly and also made available as a link from the course site. Responses for the consent form, survey, and evaluations were all collected using SurveyMonkey™ (http://www.sur- veymonkey.com). The entrance survey contained 10 questions, 8 of which were open-ended. The course evaluation contained 32 questions, 18 of which were close-ended questions required by the university. The remaining 14 questions on the course evaluation were added to speci!cally address the redesign of the course. Of these, 12 were open-ended or contained an open-ended !eld for addi- tional comments from students. Focus group sessions The focus group questions were developed using the guidelines described by Krueger and Casey (2009). For the faculty and student focus group sessions, questions were developed to gather feedback related to the three primary research goals: was this redesign effective, was it feasible, and did it impact professional identity development? Although questions were developed, a semi- structured question methodology was used to allow for further examination of feedback not anticipated by the investigators. Sessions were scheduled for 90 68 A. L. Pittenger and B. Olson-Kellogg D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 http://www.surveymonkey.com http://www.surveymonkey.com minutes, recorded with a digital audio recorder, and occurred approximately two weeks following the end of the course; full transcripts of sessions were created. Both quantitative and qualitative data sources were collected to evaluate the course redesign. Please see Table 3 for a listing of data type and source. Analysis strategy Quantitative data analysis involved the generation and interpretation of descriptive aggregate data of student scores on assignments and overall course scores in rela- tion to student performance for past course offerings when more traditional peda- gogical strategies were used. Inter-rater reliability of the grading rubric was assessed by calculating kappa value (PASW 18). Qualitative data analysis involved content analysis of both the student re"ection assignments and focus group sessions using the classic analysis strategy (Krueger & Casey, 2009), a constant comparison-like approach to reveal what participants said within their groups and allow for the evaluation of the quality and effectiveness of hypertext documents, another measure of content mastery and professional compe- tency development (Paulus, 2006). Within themes, student and physical therapy fac- ulty comments were independently reviewed by the two investigators and compared for internal consistency. Using the comparative analysis methodology described by Glaser and Strauss (1967), the focus group data were analyzed for emerging categories and themes. To obtain a mixture of female and male participants, a purposeful sampling of students from all 10 small groups was targeted. From the analysis of these data sources, the success of the process of students creating hypertext documents as an educational strategy, as well as the feasibility of demonstrating content mastery through a hypertext document, was determined. Evi- dence of transformation in both student and group identity, related to professional identity and power identity, was also evaluated. The primary research questions were evaluated using three data sources. Course scores on individual components of the project development and within- group peer review were used to evaluate the process of collaborative writing. Hypertext document quality, determined by averaging two independent rater scores (a rubric was developed and used), was used to evaluate content mastery and professional competency development. Content analysis was used to identify themes within the student re"ection assignments and focus group sessions (Krue- ger & Casey, 2009). Table 3. Data types and sources. Data type Data source Quantitative Student-ranked entrance survey and course evaluation responses Final project scores Grade distribution Qualitative Open-ended responses on entrance survey and course evaluation Student re"ection papers Student focus group session Faculty focus group session Distance Education 69 D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 Findings Quantitative data Entrance survey: background information and attitudes Of the 50 students enrolled in the 2009 summer semester offering, 76% (38 stu- dents) completed the entrance survey. Student background attitudes and experiences with online learning and collaborative projects were primarily positive. Attitude towards working as a group. Students were primarily “accepting” of work- ing in a group: 87% of students (32) responded as such; 16% of students (6) were “very excited.” Despite being generally accepting of working collaboratively in this course, students voiced a number of consistent concerns through open-ended feed- back from the entrance survey. Many students were concerned about unequal contri- butions of work by group members, their grade being impacted by the lack of work of others, poor communication within groups, and individuals in the group taking over and not allowing for cooperative learning to take place. Experience with online learning. Eighty-one percent of students (31) described pre- vious experience with online learning as “mostly positive”; 14% (6) “mostly nega- tive”; 3% (1) “very positive”; and 3% (1) “very negative.” Con!dence about course being a meaningful learning experience. Overall, 70% of students (26) were con!dent this learning experience would be meaningful; 16% (5) were “very con!dent”; and 16% (6) were “not con!dent.” Students’ concerns about taking a pharmacotherapy course online varied, but many students were concerned about the technical elements of the course and the ability to successfully complete while at a distance from the instructional team. Course evaluation: effectiveness and feasibility Of the 50 students enrolled in the 2009 offering, 64% (32) completed the course evaluation. Students generally thought the course redesign was a successful model for learning and that the course helped to emphasize the importance of pharmaco- therapy to the practice of physical therapy, but many commented that the dif!culty level of learning pharmacotherapy in this manner was greater than if a more tradi- tional pedagogical strategy had been used. Redesign is successful model for learning. Of the students who completed the course evaluation, 81% (26) felt they learned the amount expected or more than expected. Furthermore, 25% (8) felt the pace of the course was too fast, while 75% (24) felt it was just right. With reference to workload, 53% (17) felt the workload was just right, while 4% (15) felt it was too much. Overall, 81% (26) felt the struc- ture of the course was satisfactory, good, or very good. Pharmacotherapy content in physical therapy practice emphasis. The course strate- gically emphasized the importance of pharmacotherapy content for physical therapy practice. Of the respondents, 72% (23) commented that the material is either impor- tant or very important to their future practice. Final project quality and inter-rater reliability of grading rubric. Inter-rater reliabil- ity of the !nal project grading rubric was assessed by calculating a kappa value. Kappa value for the two raters was 0.196, indicating poor agreement of scores using the rubric for the two independent raters (please see Table 4 for rater scores for all 10 groups). The !nal project score for all 10 groups was determined by aver- aging the two raters’ scores. Final project scores were 82–98% and the overall !nal 70 A. L. Pittenger and B. Olson-Kellogg D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 grade distribution for the class was 48 As, 1 A!, and 1 B!. This grade distribution is consistent with grade distributions in past offerings when a more traditional peda- gogical model was used. Qualitative data Student re"ection assignments: professional identity development All student re"ection assignments were !rst reviewed by two independent reviewers for emerging themes; after several passes over the responses, four emerging themes were identi!ed and used for coding. Theme 1: evidence of collaborative learning. Consistent with the ranking and com- ments from the entrance survey, students commented on their initial reluctance with the collaborative format of this course, but described how working in a group was a valuable experience that enhanced and expanded their learning by allowing them to address an authentically complex patient scenario that they could not have suc- cessfully completed on their own: I was very hesitant going into this class with having to work with four other group members to complete this course case study. However, once we started the ongoing project together, I developed a whole new attitude about the “group work.” I soon found it a relief to be able to collaborate with others and bounce ideas off one another throughout the course. I think I would have been completely over my head if having to work on that intense of a case study [sic] on my own. Working collaboratively in a group can be both frustrating and rewarding and I feel that this semester was a little bit of both. It was frustrating at times when other group members are not participating equally and contributing less to the project. Also in group projects some people like to plan ahead, whereas others would rather procrasti- nate and wait until the last minute. This was one of the largest dif!culties I think our group encountered during the project. However, it was also a learning experience because each group member brought a different perspective and new ideas to the case study that I wouldn’t have thought of just by myself. In the end, this project has been rewarding because all together in a group we have accomplished a large project that we have worked really hard on. Collaboratively working with others in the group helped because we were able to bounce treatment ideas off of each other to best treat this individual. We all have different experiences in the past and we can bring different things to the table when Table 4. Two independent rater scores for !nal hypertext document; total possible score is 30. Group Rater 1 Rater 2 1 28 27 2 29 23 3 30 29 4 28 25 5 29 30 6 28 30 7 28 27 8 26 28 9 24 25 10 24 25 Distance Education 71 D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 treating a patient. What I think may help this patient may not be the best option and if I can use my colleagues to [provide] more suggestions I will have a better knowl- edge base to best treat my patient. This will be utilized in the future to ensure I am always striving to achieve the [best] treatment plan available for my patient. This experience taught me a great deal about communicating with others in the health-care !eld and even about communicating with individuals in a group. I have never committed myself to a project of this magnitude in a group setting and therefore had a lot of learning to do in that regard. It became very clear that simply splitting up the project and going our separate ways would not suf!ce, and I got a real apprecia- tion for the amount and quality of work that can be accomplished when a group of people gets together and bounces ideas off of one another. The group setting made me much more comfortable giving constructive feedback and critically analyzing the thoughts and ideas of other people. The feedback provided by the pharmacy and phys- ical therapy instructors paved the way for a lot of my understanding about proper communication within the health-care !eld. Many of the suggestions for improvement never even crossed my mind when creating the original draft, and I quickly learned the precision and attention-to-detail required to effectively communicate, especially when it concerns individuals with varying levels of education and understanding and requires that our thoughts and motivations be portrayed in writing. I never thought it would be so dif!cult. Theme 2: how writing impacted learning. Students stated the writing component was both helpful and dif!cult. The requirement to put their intended communication with the three audiences into text, as well as explain rationale for treatment deci- sions in writing, proved to be more challenging than students anticipated, but stu- dents acknowledged the extra struggle to put thoughts into writing was worthwhile and positively impacted their learning: Putting our ideas into writing for this project helped me think more critically about how I would actually choose to speak to each of the three audiences. We often role- play how we would speak to a patient, but so far this was our !rst experience in school where we critically thought about how we would talk to our physical therapy colleagues and the referring physician. Writing everything out also provided a unique challenge. At times it was dif!cult to detail on paper what you had in your head. I do think it was of value because being able to specify what you want to be done in writing makes you a better communicator so I did think that this was a valuable piece of this project. It was challenging to write to the different audiences. I de!nitely needed the practice in writing down my thoughts in an effective manner. I think it was a good learning experience to have to change the way we approached each different audience and made me really understand what I was talking about. By actually writing out explicitly what we were going to say to each different person made me think in a way that I was not used to. It forced me to take a step back and really think about each sentence and how that would impact the person reading the information. I am glad we had to write out our treatment plan and apply the context in words. It is easy to think about what you would do to treat a patient and that your therapy plan will "ow consistently and will make sense to their condition and not confuse the patient. When you have to write things down and make sure they "ow and all make sense together you realize things may not all be useful for a patient. It is easy to come 72 A. L. Pittenger and B. Olson-Kellogg D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 up with ideas, but sometimes too many ideas are thought of and therapy becomes overwhelming for the patient and the therapist. Theme 3: authentic assessment fostered professional identity. A primary objective of the redesign of this course was to provide students with a learning environment and experience that allowed for authentic application of pharmacotherapy content to the physical therapy practice setting. Students stated that the complexity of the learning format allowed them to take on the role of physical therapist in addressing the entire patient, both in designing physical therapy recommendations within a pharmacotherapy context, but also communicating with multiple audiences as the physical therapist. This instructional strategy seems to have allowed students the opportunity to apply the pharmacotherapy content in a meaningful and authentic way: I felt as though through this whole process I learned a lot about physical therapy inter- ventions and communications with other [disciplines]. I understood the fact that we would communicate differently with a patient than with others in the medical !eld but I was unaware our communication would vary so much between a physical therapist and a medical doctor. My future identity as a physical therapist will be impacted by this course because I really was taught how to critically think about a patient’s case and take a holistic approach to their treatment. We had to apply evidence for any treat- ment decisions utilized and we had to think about how other aspects of the patient’s case my affect their treatment. This I will utilize when I am a practicing physical ther- apist. I will make sure I have an evidence-based practice and that I am looking at vari- ous things the patient presents with, such as pharmacological agents, vitals, lab values, as well as the usual subjective items the patient identi!es. I also learned, at least partly, the process necessary to develop an entire plan of care with a holistic approach centered around speci!c patient needs and circumstances. It’s clear that in some form, every patient presents differently, and therefore every plan of care must be constructed with that in mind. I always knew that I would be an advocate for my patients, but I never really realized that the intimate contact that physical therapists have with patients relative to other health-care professionals could be used to affect so many things concerning patient care, especially when it comes to educating and informing them about all aspects of the medical care they are receiving. Negative feedback via course evaluation and student re"ection assignment Unfortunately, not all students had a positive experience with the course redesign. Five students (all within the same small group) submitted very angry and negative student re"ection statements and also provided very negative responses in the course evaluation (these are surmised to be the same students, since identical state- ments were included in both places). The comments from these students are some- what dif!cult to use, from a constructive standpoint, since they mostly include statements such as: This is the worst class I’ve ever taken. I did not learn one thing from this class. I HATED THIS CLASS!!!!!! Distance Education 73 D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 This one group (5 students out of an enrolment of 50) represented 10% of the course. The goal is for all learners to have a positive and meaningful learning expe- rience and trying to avoid such a negative experience is an ongoing issue being explored by the instructional development team. Focus group sessions Students Students who participated in the focus group session provided similar information to the course evaluation and student re"ection assignments. One new issue that emerged from the focus group session was the student desire for exams. Students stated that even though the pharmacotherapy information was readily available to them on the course Web site, they did not engage with it as much as they would have wanted, simply because they were busy and they knew it would not be assessed by an exam. Without the consequence of a lower score on an exam, they did not have the motivation to spend time with the content, even though they fully recognized the relevance and importance to their future practice. The case scenario was so complex that they did use the course content, but not as extensively as they wanted or that they anticipated they should for future practice. Another recommendation from the student focus group session was that the wiki be removed, and that all the collaborative writing occurs within the Google Sites page itself. Students felt it was cumbersome to have to move between the two sites and found that collaborative writing was possible within the Google Sites page. Faculty The focus group with the faculty primarily centered on whether or not students had demonstrated mastery of the content through this alternative assessment strat- egy and if the faculty felt it had been a successful course redesign. Overwhelm- ingly, the faculty agreed that this course design was a success. Faculty felt, after reviewing the students’ work in the Google Sites space, that students demon- strated both pharmacotherapy knowledge and, more importantly, how that knowl- edge impacted physical therapy decisions. A focus of the physical therapy program is the use of evidence-based medicine. Students were required in this project to support all recommendations with evidence. Faculty noted that this pur- poseful focus on and use of documentation was an important addition to their overall curricular focus. The requirement to explicitly cite medical evidence for both pharmacotherapy and physical therapy aspects of the patient’s care also expanded student knowledge of and ability to use reputable information outside of the physical-therapy-related medical literature, a skill development opportunity that the faculty felt was not available in other areas of the existing curriculum. At the time of these focus group sessions, the faculty could not comment on the long-term effects of students participating in this unique educational experience, since the sessions occurred prior to the start of the next semester session. How- ever, they were very pleased with the application of pharmacotherapy within a physical therapy practice scenario and were overall very happy with the evidence- based decision-making process students demonstrated within their Google Sites !nal projects. 74 A. L. Pittenger and B. Olson-Kellogg D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 When asked about the recommendation from the student focus group session for exams, the faculty supported the idea. After much discussion, the !nal recommen- dation from the faculty was to add required, individual multiple-choice exams to the current case scenario course format. To accommodate the additional course requirements, the faculty recommended limiting the audience communication focus only to the patient. Despite the bene!ts of having students practice communicating to multiple audiences, the faculty felt that the most valuable audience was the patient and the most appropriate audience given the student point of progression through the program when taking this course. Revisions and implementation in subsequent offering Based on this evidence, the following revisions were made to the course design for the 2010 summer semester offering. Five quizzes were added, although the course content remained the same, with the exception of a few updates related to therapies. The quizzes were multiple choice and administered online with a severe time restriction. Feedback from students regarding the quizzes and the course format in general was positive and the changes remained in the course for the 2011 offering. Students were asked to practice only communicating to the patient, although the examples that were provided in the !rst offering of how to adjust communica- tion to other audiences were still provided in the course materials. The recommen- dation to eliminate the wiki site and just use the Google Sites tool for both collaborative writing and the presentation of the !nal project was also implemented for this offering. Grant money to employ physical therapy PhD students for physical therapy speci!c feedback was no longer available for the 2010 offering. As an alterna- tive source of physical therapy assistance with the course, students enrolled in the 2009 offering were asked to act as teaching assistants on a volunteer basis. The familiarity of these students with the course design and the level of feed- back helpful to them when they were enrolled in the course made them ideal candidates to provide the physical therapy guidance and perspective for the 2010 offering. The grading rubric was still used for overall !nal project score for the 2010 offering, but the course director determined the !nal score. Students were provided with the option to formally request that the course director reconsider their !nal scores if there were areas the group felt were unfairly scored, but none of the group during the semester made this request. Among the original goals for the course redesign was the incorporation of dif- ferent audiences into the communication practice elements. Concern was expressed that limiting the audience to only the patient would signi!cantly diminish the learn- ing experience. Despite this change, students enrolled in the 2010 offering stated that they considered the course to be an interprofessional experience; they felt they were learning with, from, and about other professions through this course. An inter- professional aspect was never a formal consideration of this course redesign, but in retrospect, feedback from students made it apparent that they do receive information about pharmacy and the pharmacist’s role on the health-care team, as well as some experience considering the referring physician role, although these individuals are !ctional and simulated. Distance Education 75 D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 Implications: feasibility, effectiveness, professional identity development Collaborative learning and whole-task educational design was implemented in an online pharmacotherapy course delivered to doctoral-level physical therapy students. The main competency objective for students in this course was to be able to iden- tify patient and pharmacologic situations that required adjustments to the physical therapy strategies they were implementing in care settings and communicate the rec- ommendations to three levels of audience: patient, referring physician, and fellow physical therapy colleagues. Feasibility This redesign of a distance education pharmacotherapy course is a feasible educational strategy. Although the !rst offering of this strategy had grant support allowing for PhD physical therapy student assistance, the subsequent offerings were successfully implemented without this instructional support. Through the use of third-year physical therapy students providing more advanced peer review of student work in the Google Sites pages on a volunteer basis, in conjunction with the pharmacist course director review, a cost-effective way of incorporating a physical therapy validation of the application of pharmacotherapy concepts was implemented; learning technologies allowed this easy access and quick turnaround for advanced peer feedback. The Web-based collaborative tools also allowed for longitudinal learning with more advanced students revisiting the application of this content from the role of expert peer. From the perspective of course director work- load, this was also a feasible option. Teaching through the use of online resources and Web-based collaborative tools required signi!cant up-front development time, but update and revision of the learning materials and Google Sites structure have required only minimal instructor attention prior to each subsequent offering. The Web-based collaborative tools allowed the course director to interact with the students in a time-sensitive and frequent manner, but in a manageable way. Effectiveness This redesign of a completely online course was also effective at accomplishing the primary learning goals. Students demonstrated in the !nal projects that they were able to recognize and address medication-related issues speci!c to physical therapy strate- gies. Student collaboration was evident in the history tabs and through the student re"ections. A majority of students reported that this redesign provided a dif!cult, but meaningful, educational experience, and one that stretched their physical therapy knowledge and required them to apply it to an authentic physical therapy context, rather than simply memorizing mechanisms of action. Leveraging learning technolo- gies allowed for the collaborative environment, which allowed students to learn from each other as well, from the peer review of other patient scenarios in addition to working within their individual groups, further expanding the learning possible. Professional identity development Students were required to assume the role of the physical therapist through this educational experience and communicate to various audiences through that voice, rather than the voice of a physical therapy student taking a pharmacy course. 76 A. L. Pittenger and B. Olson-Kellogg D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 Through the use of digital collaborative writing, students also developed professional identities and competencies as they practiced “ways of acting, ways of representing, and ways of being” (Fairclough, 2004, p. 228). Students stated overwhelmingly that they, for the !rst time in their curriculum, had to address the patient as a whole per- son, not a system requiring attention, and with the authority of a practicing physical therapist. Students reported that this was a powerful identity shift and they were happy to have been able to do it in a collaborative way, noting that individually they would have been overwhelmed and this course provided an opportunity to practice this assumed role in a safe, simulated, and supported environment. Traditional classroom pedagogical strategies and assessment methods, such as lecture-based resources and multiple-choice exams administered face-to-face or online, have been used over the past !ve years to demonstrate mastery of the con- tent of this course. Students successfully completed the course by achieving passing scores on the exams, which focused on pharmacology mechanism. However, these pedagogical and assessment tools were not authentic learning or assessment strate- gies, as students in this professional role will not administer medication, nor in most cases, provide pharmacotherapy recommendations. Because physical therapists gen- erally spend an extended period of time with patients, they play an integral role in encouraging patient adherence, recognizing adverse events, adjusting physical ther- apy regimens to accommodate drug therapy and patient speci!c characteristics, as well as providing a communication bridge between the patient, referring physician, and fellow physical therapists. Therefore, instead of assessing students’ abilities to memorize pharmacology information, assessment of physical therapy students’ abili- ties to effectively communicate evidence-based strategies that incorporate pharmaco- therapy concepts with the patient, attending physician, and physical therapist audiences serves as a much stronger educational tool, mirroring future practice. Physical therapy students must not only practice identifying issues and making the appropriate clinical judgments, but also communicating these ideas and supportive reasoning. Without effective communication and collaboration skills and the ability to adjust messages to various audiences, quality health care is not delivered. Limitations Although this course redesign has successfully created a learning experience that allowed students to apply pharmacotherapy content to a speci!c practice experience, how overall physical therapy competency was impacted is currently unknown, nor is there evidence that students approached subsequent courses in new ways. As in previ- ous face-to-face and traditional educational approaches, students adequately achieved the learning outcomes of the course, but the role successfully meeting educational goals played in creating more clinically competent and adaptable practitioners remains unknown. Proactively building extensions from this experimental approach to evaluate the overall impact of integrating content and application after the three implementa- tions of this redesign was a missed opportunity, but the redesign strategy needed to be tested before connections to performance outcomes could be measured. Future research Evaluating the impact of this course strategy and use of learning technologies beyond the individual course is an area of current research exploration. Speci!cally, Distance Education 77 D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 ways of threading additional pharmacotherapy application opportunities throughout the physical therapy curriculum and measuring clinical performance while at prac- tice sites is currently underway; the impact of using authentic application opportuni- ties early and throughout the curriculum on practice performance, something that is of interest to educators in all areas of focus, is being measured. The grading rubric used in this course proved to have too much subjectivity, resulting in poor inter-rater reliability. Despite the lack of consistency between potential graders, the rubric has continued to be used by the course director as the sole !nal project grader. Developing a grading rubric with a high inter-rater reliabil- ity would allow grading to be shared among others, minimizing the grading work- load associated with this course. Conclusion Evaluation of the process of collaboratively creating hypertext documents for multi- ple audiences, as well as the end product of a hypertext document as a demonstra- tion of content mastery, served to further develop this tool for future offerings and other courses offered using a distance education format. In this example, learning technologies allowed a more learner-centered educational strategy and authentic means to acquire content knowledge and assess student mastery. Application and evaluation of these Web-based collaborative tools contributed to evidence-based writing strategies implemented in the classroom as a means for students to demon- strate mastery of content and effective communication skills, imperative to the health-care profession. The !ndings demonstrate that the implementation of collaborative digital writing with a hypertext document case scenario assessment as the primary assessment tool in this online pharmacotherapy course delivered to doctoral-level physical therapy students is a feasible and effective educational strat- egy. The description of this course redesign process, grounded in educational theory and guided by instructional design principles, can hopefully aid educators who wish to use learning technologies to create distance education environments and experiences that are meaningful and authentic. Acknowledgements This study was supported by an Interdisciplinary Studies of Writing grant from the Center for Writing at the University of Minnesota. The authors acknowledge the expert editing assistance of Amy LimBybliw, MA, in preparing this manuscript. Notes on contributors Amy L. Pittenger is the Director of Interprofessional Education and assistant professor of pharmaceutical care and health systems in the College of Pharmacy, University of Minnesota, USA. Becky Olson-Kellogg is the residency director of the Geriatric Clinical Residency Program in Physical Therapy and an assistant professor of physical therapy in the School of Medicine, University of Minnesota, USA. References Alverman, D. E., O’Brien, D. G., & Dillon, D. R. (1996). Conversations: On writing qualita- tive research. Reading Research Quarterly, 31, 114–120. Retrieved from http://www. jstor.org/stable/748242?origin=JSTOR-pdf 78 A. L. Pittenger and B. Olson-Kellogg D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 http://www.jstor.org/stable/748242?origin=JSTOR-pdf http://www.jstor.org/stable/748242?origin=JSTOR-pdf Amhag, L., & Jakobsson, A. (2009). Collaborative learning as a collective competence when students use the potential of meaning in asynchronous dialogues. Computers & Educa- tion, 52, 656–667. doi: 10.1016/j.compedu.2008.11.012 Baartman, L. K. J., Bastiaens, T. J., Kirschner, P. A., & van der Vleuten, C. P. M. (2007). Evaluating assessment quality in competence-based education: A qualitative comparison of two frameworks. Educational Research Review, 2, 114–129. doi: 10.1016/j.edu- rev.2007.06.001 Bean, J. C. (2001). Engaging ideas: The professor’s guide to integrating writing, critical thinking, and active learning in the classroom. San Francisco, CA: Jossey-Bass. Berninger, V. W., & Winn, W. D. (2006). Implications of advancements in brain research and technology for writing development, writing instruction, and educational evolution. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 96–114). New York, NY: Guildford Press. Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press. Cole, M. (2009). Using wiki technology to support student engagement: Lessons from the trenches. Computers & Education, 52, 141–146. doi: 10.1016/j.compedu.2008.07.003 Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Qual- ity of Life Research, 12, 229–238. doi: 10.1023/A:1023254226592 Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage. Creswell, J. W. (2010). Mapping the developing landscape of mixed methods research. In A. Tashakkori & C. B. Teddlie (Eds.), Handbook of mixed methods in social & behavior research (pp. 45–68). Thousand Oaks, CA: Sage. Dillman, D. A., Tortora, R. D., & Bowker, D. (1998). Principles for constructing web sur- veys (SESRC Technical Report). Pullman: Washington State University. Retrieved from http://www.websm.org/uploadi/editor/Dillman_1998_Principles_for%20Constructing.pdf Doering, A., & Veletsianos, G. (2008). What lies beyond effectiveness and ef!ciency? Adventure learning design Internet and Higher Education, 11, 137–144. doi: 10.1016/j. iheduc.2008.07.004 Duffy, T. M., & Cunningham, D. J. (1996). Constructivism: Implications for the design and delivery of instruction. In D. H. Jonassen (Ed.), Handbook of research for educational com- munications and technology (pp. 170–198). New York, NY: Macmillan Library Reference. Fairclough, N. (2004). Semiotic aspects of social transformation and learning. In R. Rogers (Ed.), An introduction to critical discourse analysis in education (pp. 225–235). Mah- wah, NJ: Lawrence Erlbaum Associates. Gaddis, S. E. (1998). How to design online surveys. Training and Development, 52(6), 67– 71. Retrieved from http://www.astd.org/TD/ Gee, J. P. (2003). What video games have to teach us about learning and literacy. New York, NY: Palgrave. Gee, J. P. (2005). An introduction to discourse analysis theory and method (2nd ed.). New York, NY: Routledge. Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine. Hull, D. M., & Saxon, T. F. (2009). Negotiation of meaning and co-construction of knowl- edge: An experimental analysis of asynchronous online instruction. Computers & Educa- tion, 52, 624–639. doi: 10.1016/j.compedu.2008.11.005 Jonassen, D. H. (2000). Computers as mindtools for schools: Engaging critical thinking (2nd ed.). Upper Saddle River, NJ: Prentice Hall. Keller, J. M. (1987). Development and use of the ARCS model of instructional design. Jour- nal of Instructional Development, 10(3), 2–10. Retrieved from http://www.jstor.org/sta- ble/30221294?origin=JSTOR-pdf Keller, J. M. (2008). First principles of motivation to learn and e3-learning. Distance Educa- tion, 29, 175–185. doi: 10.1080/01587910802154970 Kramarae, C. (2007). Gender matters in online learning. In M.G. Moore (Ed.), Handbook of distance education (pp. 169–180). Mahwah, NJ: Lawrence Erlbaum. Krueger, R. A., & Casey, M.A. (2009). Focus groups: A practical guide for applied research (4th ed.). Thousand Oaks, CA: Sage. Distance Education 79 D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 http://www.websm.org/uploadi/editor/Dillman_1998_Principles_for%20Constructing.pdf http://www.astd.org/TD/ http://www.jstor.org/stable/30221294?origin=JSTOR-pdf http://www.jstor.org/stable/30221294?origin=JSTOR-pdf Landow, G. P. (2006). Hypertext 3.0: Critical theory and new media in an era of globaliza- tion. Baltimore, MD: Johns Hopkins University Press. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. New York, NY: Cambridge University Press. Lewis, C., & Ketter, J. (2004). Learning as social interaction: Interdiscursivity in a teacher and researcher study group. In R. Rogers (Ed.), An introduction to critical discourse analysis in education (pp. 117–146). Mahwah, NJ: Lawrence Erlbaum. Lund, A. (2008). Wikis: A collective approach to language production. ReCALL, 20, 35–54. doi: 10.1017/S0958344008000414 Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43–59. doi: 10.1007/BF02505024 Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage. Paulus, T. M. (2005). Collaborative and cooperative approaches to online group work: The impact of task type. Distance Education, 26, 111–125. doi: 10.1080/015879105000 81343 Paulus, T. M. (2006). Challenge or connect? Dialogue in online learning environments Jour- nal of Computing in Higher Education, 18, 3–29. doi: 10.1007/BF03032722 Rourke, L., & Kanuka, H. (2009). Learning in communities of inquiry: A review of the liter- ature. Journal of Distance Education, 23(1), 19–48. Retrieved from http://www.jofde.ca/ index.php/jde/index Sosnoski, J. (1999). Hyper-readers and their reading engines. In G. Hawisher & C. Selfe (Eds.), Passions, pedagogies and 21st century technologies (pp. 161–177). Logan: Utah State University Press. Steiner, W. A., Ryser, L., Huber, E., Uebelhart, E., Aeschlimann, A., & Stucki, G. (2002, November). Use of the ICF model as a clinical problem-solving tool in physical therapy and rehabilitation medicine. Physical Therapy, 82, 1098–1107. Retrieved from http:// www.ptjournal.org/ Tennyson, R. D., & Breuer, K. (2002). Improving problem solving and creativity through use of complex-dynamic simulations. Computers in Human Behavior, 18, 650–668. Retrieved from http://elsevier.com/locate/comphumbeh Trentin, G. (2009). Using a wiki to evaluate individual contribution to a collaborative learn- ing project. Journal of Computer Assisted Learning, 25, 43–55. doi: 10.1111/j.1365- 2729.2008.00276.x van Merriënboer, J. J. G. (2007). Alternate models of instructional design: Holistic design approaches and complex learning. In R.A. Reiser & J.V. Dempsey (Eds.), Trends and issues in instructional design and technology (pp. 72–81). Upper Saddle River, NJ: Pear- son Education. Wang, Q. (2008). A generic model for guiding the integration of ICT into teaching and learning. Innovations in Education and Teaching International, 45, 411–419. doi: 10.1080/14703290802377307 Weedon, C. (1997). Feminist practice and poststructuralist theory (2nd ed.). Oxford: Blackwell. 80 A. L. Pittenger and B. Olson-Kellogg D ow nl oa de d by [U ni ve rs ity o f M in ne so ta L ib ra ri es , T w in C iti es ], [A m y L . P itt en ge r] a t 0 6: 18 3 0 A pr il 20 12 http://www.jofde.ca/index.php/jde/index http://www.jofde.ca/index.php/jde/index http://www.ptjournal.org/ http://www.ptjournal.org/ http://elsevier.com/locate/comphumbeh