key: cord-0766103-e2coy5ms authors: Matsuzaw, Yoshiaki; Oshima, Jun; Oshima, Ritsuko; Niihara, Yusuke; Sakai, Sanshiro title: KBDeX: A Platform for Exploring Discourse in Collaborative Learning date: 2011-12-31 journal: Procedia - Social and Behavioral Sciences DOI: 10.1016/j.sbspro.2011.10.576 sha: 7bf2dd1d23ee0fc1d363812d9ae02157e29053c8 doc_id: 766103 cord_uid: e2coy5ms Abstract Knowledge building as defined in this study is emergent collaborative learning on ill-structured tasks. Although discourses in collaborative learning have been analyzed with traditional qualitative approaches in the learning sciences field, it is difficult to capture the group dynamics. Hence, we are trying to establish a methodology for discourse analysis in collaborative learning from the perspective of complex network science. In order to conduct this study effectively, we are currently developing an application platform, called Knowledge Building Discourse Explorer (KBDeX). The goal of this project is not only to facilitate productive communication between researchers who are concerned with research on knowledge building or emergent collaborative learning, but also to encourage students to explore their own group dynamics by themselves. KBDeX is an analysis platform to visualize network structures of discourse based on the bipartite graph of words × discourse units. KBDeX can visualize them into three different network structures of: (1) students, (2) discourse units, and (3) selected words. The users can explore these three networks with its coefficients and analyze the discourse across phases or in a and stepwise way. Using discourse which has been already analyzed with a traditional qualitative approach, we will demonstrate the beneficial attributes of the KBDeX platform. Knowledge building as defined in this study is emergent collaborative learning on ill-structured tasks in the classroom (Scardamalia & Bereiter, 2005) . As present, it is difficult to capture its group dynamics with existing analysis approaches. In this section, we discuss the unique nature of knowledge building by comparing it with ordinary collaborative learning, and explore why current approaches to discourse analysis might fail to capture its nature. Successful collaborative learning in the classroom is often well structured, with fixed-size, small groups involved in some challenging tasks across specific periods of time. In cases like scripted cooperation (O'Donnell, 1999) or reciprocal teaching (Rosenshine, & Meister, 1994) , students' roles are specified in efforts to create the ideal situation for collaboration. Knowledge building, on the other hand, is quite opposite to this stream. Students are encouraged to collaborate with one another in a flexible manner even though they are allocated into groups. The time schedule should not be fixed because of the emergent nature of the learning, and we do not expect students to see an end to their learning. How do we approach assessing this emergent nature of knowledge building or collective knowledge advancement? Thus far, researchers have applied three methodological approaches to capture the nature of knowledge building. The first approach establishes the rubrics of content knowledge that researchers expect the learners to acquire following their learning. The researchers can then identify whether learners' knowledge advances or not based on their established rubrics. The second approach involves researchers in analyzing the process of learning by breaking it into small units to categorize into different cognitive actions. This allows them to identify patterns or models of cognitive processes that the students engage in (e.g., van Aalst, 2009) . The third approach is fine-grained discourse analysis performed as a case study, which helps researchers describe what is happening in students' collaborative learning (Oshima et al., 2006) . The combination of discourse analysis and narrative is a popular methodology for analyzing the classroom environment. Although the three approaches are appropriate to discuss well-structured collaborative learning, none of them are sufficient to capture collective knowledge advancement. Regarding the content of knowledge, we are not only concerned with deep comprehension of domain-specific knowledge but also epistemic operation by learners to advance their collective knowledge. Since the epistemic operation is a process, a static evaluation of knowledge will not capture its dynamics. Categorization of the cognitive processes that learners engage in might capture the epistemic operation, but it is so content-free that we cannot describe what knowledge learners actually develop. Consequently, fine-grained discourse analysis with narratives would be the last option. Although the microscopic view of discourse analysis provides us with details about how learners develop their collective knowledge within a period of time, we also need to describe its macroscopic view so that we can verify why the detected microscopic discourse should be important to argue and how the detected pieces of discourses are placed in the macro structure of collaboration. In this paper, we propose the network structure analysis of discourse and its supporting tool as a macroscopic analysis of knowledge building. In the next section, we briefly review complex network science research and how the approach can be applied to learner discourse on Knowledge Forum ® , a computer-supported collaborative learning environment. In section 3, we describe the methodology we are currently developing for analyzing discourse from the macroscopic view with describing a platform application for the analysis, which we call Knowledge Building Discourse Explorer (KBDeX). In section 4, we demonstrate an example of the application of the method using data which have been already analyzed. Finally, we discuss the results of the application in section 5. Complexity is a key concept in 21st century science (Watts, 2007) . Agents having a variety of resources are linked to one another in multi-layer communication tools. The network structure created through such communication generates new insights and creates new knowledge (Davis, & Sumara, 2006; Scardamalia, & Bereiter, 2005) . Complex network science (e.g., Strogatz, 2001 ) is based on an analytic approach to describing a variety of network structures developed by statistical physics (e.g., the smallworld and scale-free network structures). Oshima et al. (2007) argued that the complex network science approach was useful for evaluating collective knowledge advancement in computer supported collaborative learning, and demonstrated how the complexity system based on discourse on genetically modified foods by 5th grade students was different from that by experts. Zhang et al. (2009) used social network analysis to visualize and compare classroom collaboration among 4th graders on Knowledge Forum ® across three years, and concluded that knowledge advancement was facilitated when the participation structure was more distributed. These studies were focused on the issue of how to visualize the macro structure of collaboration appropriately, and were not concerned with its relation to conventional approaches to discourse analysis. In this study, we consider how to make the macroscopic discourse analysis informative for the microscopic discourse analysis. Particularly, we attempt to focus our attention on the individual differences in learners' contributions to discourse in collaborative learning. We are trying to establish a methodology for discourse analysis in collaborative learning from the perspective of complex network science. In order to conduct this study effectively, we are currently developing an application platform, called Knowledge Building Discourse Explorer (KBDeX). The goal of this project is not only to facilitate productive communication between researchers who are engaged in research on knowledge building or emergent collaborative learning, but also to encourage learners to explore their own group dynamics by themselves. KBDeX is an analysis platform to visualize network structures of discourse based on a bipartite graph of words × discourse units (e.g., conversation turns, postings on Bulletin Board System, and sentences). Our current version of the methodology used to analyze learning discourse with KBDeX is described in this section. KBDeX users must prepare discourse data in comma separated format (.csv) and a list of words in general text format (.txt) that you want to select for creating a bipartite graph. Currently, there are two ways to get the text discourse from the collaborative learning situation. One way is a transcription from camcorder recorded sources, another way is recorded logs by online CSCL systems. Current version of the software can retrieve the data automatically from Knowledge Forum ® . However the raw data from both ways are not well-structured to be accepted by the software; hence, it is necessary for the user to perform some processes to make the data appropriate to be accepted by the system. The processes include: 1) to merge the singular and plurals of nouns 2) to merge different words which have the same meaning (ex. "net" and "internet") 3) to merge the different conjugated forms of verbs if the user needs to analyze the verb 4) to remove unnecessary discourse units from the raw data. These processes require qualitative considerations; therefore the processes are currently performed by users. The supporting tools for these processes are in development. The second qualitative process for users to prepare data is target word selection. The target words are selected by experts in consideration of the learning objectives. There are criteria used to select the words which are considered to be important: 1) to learn the subject-matter 2) to manage learning (it is called epistemic words). Only nouns are selected in the current version of our method. A simple supporting tool (Fig 1) for this process is built in KBDeX. The word selection tool provides us with three views 1) word selection text area (Left in Fig 1) , 2) word frequency view (Center in Fig 1) , and 3) the discourse view (Right in Fig 1) . In the view of 2) and 3), the selected words are indicated in red. KBDeX builds three different network structures from the data and show them with a discourse viewer (shown in Fig 2) . The main view of KBDeX has four windows: (1) The discourse viewer which shows an overview of the discourse and selected word (top left window in Fig 2) , (2) the network structure of students (top right window in Fig 2) , (3) the network structure of discourse units (as note in the Knowledge Forum ® ) (bottom left window in Fig 2) , and (4) the network structure of selected words (bottom right window in Fig 2) . The networks of notes (3) and words (4) are created by the bipartite network of notes × words; each of them is shown as a one-mode projection of a bipartite network. The network of students (2) is also a one-mode projection of the words × students bipartite network. Although the users can select the layout algorithm of these networks, in the default, the circular layout is used for the students and notes network, and Fruchterman-Reingold layout is used for word network. Then the nodes in the note network are sorted by the time scale. The users can explore these three network structures seamlessly by clicking on a particular node, and then the system indicates the relative nodes in the other two networks. The color of the selected nodes and relative nodes in the other network are changed from yellow to red. This is demonstrated in Fig 2 and 3 . If the user selects a particular student in the students network, then the notes written and the words used by the selected student are in red (Fig 2) . If the user selects a particular word in the words network, then the notes that include the selected word and the students that used the selected word are indicated (Fig 3) . Using the discourse control buttons which are located in the discourse viewer, KBDeX displays how the three network structures are developed across an animated evolution through the discourse. This function is useful for users to detect pivotal points in the discourse so that they can focus their attention on such points. KBDeX can measure a variety of coefficients used in complex network science, such as those for betweeness centrality (BC), degree centrality, and closeness centrality. The selected coefficients are dynamically plotted to a chart along with the progress of the discourse (Fig 4) . KBDeX has a function that the user can make the particular nodes in any network inactive. The deactivated nodes are shown in gray color, and then the bipartite connections in all networks is faded ( Fig 5) . This function is used for two types of analysis that is described in the following paragraphs. The first purpose is for phase analysis. This is used to compare coefficients (the mean BCs of notes are used in the demonstration in section 4) across different phases of collaborative learning between groups. The user can do it by activating the notes in the notes network by time phase. The second purpose is for stepwise analysis (Oshima et al., 2007) . Stepwise analysis is to calculate each individual's contribution. In the stepwise analysis of network structure, each student's contribution was calculated as the mean absolute values of changes in the BCs of words when the target student's written discourse was completely omitted from the data set. In other words, the user compare the BCs of words representing the network structure of the target student's discourse with the network structure not including the target student's discourse as data. The user can do it by deactivating the particular student who is the target of the analysis. The participants were two classes of secondary school students, from a 10th grade course on career preparation and inquiry (n=21) and an 11th grade course focusing on computers and their impact on "global society" (n=19). The courses were taught concurrently by the same teacher at an inner city school in Western Canada. The topics posed to the students were "Outbreaks of Severe Acute Respiratory Syndrome (SARS)" and "Avian Flu in 2003 and 2004." The students could build on their knowledge of science by studying what was known about these phenomena or they could, for example, critique media attention, examine the economic impact, or form a position on how governments should have responded to the outbreaks. The second main topic on the 11th grade course syllabus was "computer viruses," which was added to SARS and Avian Flu topics as a third main topic for inquiry, with the aim of having the students examine the nature of viruses in both biological and non-biological systems and identify patterns across them. The two classes shared a Knowledge Forum ® database and worked on the same topics. To limit the number of notes they would encounter, the students were divided into four groups. Each group had students from both classes, with an equal number of students from each class; the students could choose their own groups but the teacher made some minor changes. Each group shared its own views on Knowledge Forum ® , and the groups were not expected to interact with each other during the inquiry. Based on an analysis of discourse types students engaged in, van Aalst (2009) identified four groups: Group A was most consistent with knowledge building, Group D was sharing knowledge but was not frequently engaged in knowledge building, and Groups B and C had mixed characteristics of knowledge building and knowledge sharing. In this study, we attempted to examine collective knowledge advancement in Groups A and D with our developing network structure analysis, for the purpose of exploring how informative the network structure analysis is in identifying profiles of collaborative learning discourse by students who engaged in different types of practices. We selected 409 nouns as indicators of student understanding. The agreement between two independent experts who selected the words was over 80%, and their disagreements were resolved through discussion. After selecting the words, we created networks using KBDeX. First, we examined group differences in discourse between Groups A and D by comparing the network structures of notes (Fig 6) . We compared the mean BCs of notes across three different phases of collaborative learning between groups. A 2 (Group) × 3 (Phase) ANOVA on the mean BCs showed an interaction effect, F(3,198) = 9.7098, p < 0.01, indicating that there were no significant differences in mean BCs across phases in Group A, but BCs significantly decreased across phases in Group D (Fig 7) . Second, we examined individual student's contributions to the knowledge advancement by analyzing the network structure of words (Fig 8) . We calculated each individual's contribution by using stepwise analysis in KBDeX. The results of the analysis is shown in Fig 9. An 11 (Student) × 3 (Phase) ANOVA on the mean absolute values of changes in the BCs revealed an interaction effect, F(20, 374) = 7.06, p < .01, indicating that different students contributed to the discourse in different phases in Group A. On the other hand, a 12 (Student) × 3 (Phase) ANOVA revealed a significant effect both for Students, F(11, 204) = 2.21, p < .05, and for Phases, F(2, 408) = 17.60, p < .01, indicating that students had quite similar contribution patterns, with the exception of one student (Student #7). The network structure analysis revealed remarkable differences between knowledge building and knowledge sharing groups. First, the knowledge building group was engaged in collective knowledge advancement in a quite stable manner across the three phases, suggesting their continuous involvement in collective knowledge advancement. Second, contributions by students in the knowledge building group were divergent across the phases. Different individuals contributed in different ways at different phases, which suggests that the organization of inquiries might be distributed across individuals and its structure made them contribute to their knowledge advancement in unique ways. KBDeX can support several parts of these analysis procedures effectively. The demonstration example described in section 4 has already been completed by the co-authors (Oshima et al., 2010) with general purpose network analysis tool (Pajek). Through the process of double-checking the data using KBDeX (described in this paper), the advantage of the KBDeX software over Pajek was indicated. The analysis was completed for two to four hours using KBDeX, it took two to four days using the general purpose tool. We designed KBDeX to be easy enough for all learners to use. In the future, we plan to let students use this software. Complexity and Education: Inquiries into learning, teaching, and research Structuring dyadic interaction through scripted cooperation Cognitive perspectives on peer learning Complex network theory approach to the assessment on collective knowledge advancement through scientific discourse in CSCL Network structure analysis approach to collaborative learning as a complex system Knowledge-building activity structures in Japanese elementary science pedagogy Reciprocal teaching: A review of research Distributed Cognitions: Psychological and Educational Considerations Does education for the knowledge age need a new science? Distinguishing knowledge sharing, knowledge construction, and knowledge creation discourse Designs for collective cognitive responsibility in knowledge-building communities The platform application development project has been just started, and the current version of the KBDeX has been developed as a prototype. Therefore of course the tool has a number of problems for performance, usability, and scalability. Currently, the most time-consuming and skilled process of this analysis method is the data creation of the well-structured data. Enhanced support of the data creation process, the software, and the analysis method are our further consideration.