Author accepted manuscript April 2016 1 Learning through engagement: MOOCs as an emergent form of provision Authors Sukaina Walji Corresponding Author University of Cape Town (UCT) Private Bag Rondebosch 7700 Tel: +27 21 650 4067 Sukaina.Walji@uct.ac.za Andrew Deacon University of Cape Town Tel: +27 21 650 5034 Andrew.Deacon@uct.ac.za Janet Small University of Cape Town Tel: + 27 21 650 4469 Janet.Small@uct.ac.za Laura Czerniewicz University of Cape Town Tel: + 27 21 650 5036 Laura.Czerniewicz@uct.ac.za Disclosure: The authors have no conflict of interests with regards to authorship of this paper. Author accepted manuscript April 2016 2 Abstract Massive Open Online Courses (MOOCs) are a new form of educational provision occupying a space between formal online courses and informal learning. Adopting measures used with formal online courses to assess the outcomes of MOOCs is often not informative because the context is very different. The particular affordances of MOOCs shaping learning environments comprise both scale (in terms of numbers of students) and diversity (in terms of the types of students). As learning designers we focus on understanding the particular tools and pedagogical affordances of the MOOC platform to support learner engagement. Drawing on research into learner engagement conducted in the broader field of online learning, we consider how learner engagement in a MOOC might be designed for by looking at three pedagogical aspects: teacher presence, social learning and peer learning. Keywords: MOOCs, online learning design, peer learning, student engagement, teacher presence Author accepted manuscript April 2016 3 Introduction MOOCs occupy an “in-between” space – neither traditional formal course nor online textbook – providing an opportunity for large numbers of people to become engaged through forms of social interaction. Yet simply using a platform that promotes social learning is, of course, not enough for engaged learning to happen. Drawing on research into learner engagement conducted in the broader field of online learning, we consider how learner engagement in a MOOC might be designed for by looking at three pedagogical aspects: teacher presence, social learning and peer learning. We explore learner engagement in two MOOCs developed at the University of Cape Town as a way of interpreting and analysing how people respond to learning design choices and are likely to be learning through this engagement. Our focus is specifically concerned with the roles of educators, how cohorts of learners respond and how individual learners engage with each other. Our interest is in exploring what constitutes engagement in a MOOC and what factors encourage MOOC participants to engage in order to support their learning. Lack of engagement on MOOCs may be due to factors such as connectivity, digital skills, time zones and institutional power dynamics. As learning designers, our focus is rather on understanding the particular tools and pedagogical affordances of the MOOC platform to support learner engagement. MOOCs provide opportunities for pedagogical exploration especially because their place in the non-formal part of the higher education landscape (Figure 1) does yet do not threaten formal course provision whether online or face-to-face (Czerniewicz et al, 2014). Nonetheless, the space is a fast evolving one and with a number of MOOC variants emerging, MOOCs may come to be potential threats to traditional offerings in the future should alternative forms of certification prove to be acceptable to employers (de Freitas et al., 2015). It is therefore an opportune moment to be studying engagement. Author accepted manuscript April 2016 4 Figure 1: Higher education course provision landscape Designing MOOCs MOOCs and formal online courses Learners approach MOOCs differently than if they are taking a formal registered course. As MOOCs are not formal online courses resulting in the award of credits, they require no upfront payment, nor do they have a requirement for prior learning or qualifications, and learners who enrol may have a very wide range of motivations for joining a course and their conception of what might constitute success for them will similarly vary. While for some learners completing the course in order to be eligible for a certificate may constitute success, for others it might be to sample materials for general interest or audit a course to learn one or two new things with no intention of completing the course (Liyanagunawardena et al., 2015). A recent study analysing 86 MOOCs offered by MIT and Harvard on the edX platform indicated that some 39% MOOC learners were in fact other educators who were interested in learning how another educator taught a subject or just to experience how to teach and design for an online environment (Ho et al., 2015). Author accepted manuscript April 2016 5 Attributes of MOOCs The particular affordances of MOOCs shaping learning environments comprise two aspects: the scale (in terms of numbers of students) and the diversity (in terms of the types of students). These aspects constrain possible learning designs and shape how students might engage, but these aspects also provide opportunities that leverage the affordances of scale and diversity. Adopting measures used with formal online courses to assess the outcomes of MOOCs is often not informative because the context and objectives are typically very different. Suggesting successful learning in MOOC should be measured as completion means all MOOCs perform poorly, which is not necessarily the case. MOOC designers and educators have sought other indicators and measures for what might constitute a successful MOOC. Book publishers, television producers and public lecture presenters would use sales, views and attendees in much the same way MOOCs used enrolment as a show of interest or engagement. Such indicators capture many possible motivations. Identifying some of the main reasons for why participants enrol in MOOCs then becomes an important focus. Understanding these indicators in context allows for course design teams to work with educators to develop strategies for designing MOOCs, as well as assisting institutions to decide what sort of courses and topics might be particularly suitable for MOOCs. There are additional measures too that are more similar to those of formal courses. In addition to completion rates, Yuan and Powell (2013) suggest issues of quality, sustainability, awarding of credit and pedagogy of MOOCs as major concerns. While completion and pedagogy relate directly to learning design, the remainder are much more strongly linked to external decisions and are beyond the scope of this paper. Quality assurance is clearly needed to ensure that they conform to effective practice (Jona and Naidu, 2014), but such processes exist outside the design of individual courses. Engagement as a lens We use the term “engagement” to describe the willingness and extent to which people are active in a MOOC as displayed through their interaction with the content and people in the course. Student engagement can be seen a measure of “the investment of time, effort and other relevant resources by both students and their institutions intended to optimise the student experience and enhance the learning outcomes and development of students, and the Author accepted manuscript April 2016 6 performance and reputation of the institution” (Trowler, 2010, p.3). This could encompass multiple activities such as students studying a subject, obtaining feedback, taking part in active and collaborative learning, interacting with educators and other students with an understanding that greater levels of engagement lead to a richer learning experience (Robinson & Hullinger, 2008). Student engagement is considered to be one of the “primary components of effective online teaching” (Dixson, 2010, p.1), is seen as important for retention (de Freitas et al., 2015) and enhances the quality of the overall student experience. We acknowledge the term ‘engagement’ encompasses a range of dimensions and understandings incorporating behavioural, psychological and socio-cultural aspects of learner and institutional experiences (Kahu, 2013; Hew, 2014; Trowler, 2010). Learning design for MOOCs For course designers, the particular characteristics of open participation enabled by MOOCs leads to issues of scale and participant heterogeneity; there is a difference in designing for a small group of known students with similar backgrounds and designing for diverse participants and to a massive scale. Specific implications of this manifest in considering the role of the teacher and the constraints or opportunities afforded by scale. Rather than seeing MOOCs as inferior to to formal online courses, we explore what makes MOOCs unique and what affordances MOOCs have that formal online courses do not. Specifically our focus is on the “massive” in terms of numbers enrolled and “open” in terms of open to diverse types of participants. Knox (2014) contends that that “massiveness is more than simply large enrolment numbers” and that it constitutes “not only something unprecedented in education, but also something of significant value to continued work in an educational domain that is becoming increasingly global in its capacity and reach” (p. 165), while Stewart suggests that this massiveness has the potential to expose students to the practice of networked learning (Stewart, 2013). Building on these two perspectives, we are interested in exploring what it is that MOOCs could do well through careful design, that cannot easily be achieved in formal online courses. Author accepted manuscript April 2016 7 Methods Two courses are used as for case studies to observe learners’ digital experience while learning online and we purposefully look for evidence where our attempts at promoting learner engagement through learning design succeeded or fell short of the intention. We have used a mixed methods approach, although our primary lens is a thematic analysis of the online discussion forums. MOOC research looking for generalised learners’ engagement has used learner activity data, often referred to as clickstream data to calculate measures such as completion and drop-outs (Veletsianos, Collier and Schneider, 2015). While this provides a high level view of learner behaviour, to gain a deeper understanding of what learners do and say we must look more closely to learners’ voices as legitimate evidence to understand engagement. There are multiple ways in which a learner could be engaging including vicarious learning, direct learning from others, implicit learning knowledge sharing and conversational learning (Ferguson et al., n.d.) and understanding how and why learners engage in particular activities provides a more nuanced view of how learners experience MOOCs. Observation helps us achieve this aim as it enables rich description and the opportunity to draw inferences about someone’s perspectives that could not be obtained by relying on for, example, interview data (Maxwell, 2005). In our case observation enables us to draw inferences of learners’ actual experiences in real time, that is while they were participating in the MOOC. Since the courses are online we cannot directly observe the learners physically, rather we are seeing what they do online. We use four sources of data to explore how learners engaged and responded to the course’s learning design: ● Platform activity data (for example, the impact on participation of sending emails from educators), ● Interviews with the educators, ● Learner post-course surveys ● A selection of in-course comments from learners themselves left in the course discussion spaces. We are particularly looking for evidence of how the learners experience and engage in the learning process in their own words, and use other sources of data to draw out, confirm or Author accepted manuscript April 2016 8 inform learners’ responses to design. Thus we use the learning analytics data to confirm what is typical and help highlight what is unusual in how learners engage. While the learning analytics data do not provide evidence of the experiential aspects of learning, they can provide some clues of how to make sense of our observations of learners. In addition, we we have coded the online discussions from the forums for indicators of our three key concepts: teacher presence, social learning and peer learning. We differentiated negative and positive comments, and extracted comments that garnered significant ‘likes’, as this indicates broader agreement among the learner community for that view, whether positive or negative. We see negative comments as a form of engagement and as learning designers are keen to understand the difficulties and confusion that learners may experience. Case study: Designing for engagement in MOOCs The two MOOCs studied were six-week courses run on the FutureLearn platform during 2015, ‘Medicine and the Arts’ and ‘What is a Mind?’. Medicine and the Arts: Humanising Healthcarei is an interdisciplinary course introducing the emerging field of medical humanities. Convened by an anthropologist (Associate Professor Susan Levine) and a medical doctor (Professor Steve Reid), the course assembled 16 disciplinary experts across health sciences, social sciences and the arts to bring their perspectives into dialogue about six topics selected by the lead academics. The course was run twice during 2015 and data from both runs is included in this paper. What is a Mind?ii led by neuroscientist, Professor Mark Solms, explored scientific and philosophical concept related to understanding human minds. The course brings in perspectives from a range of disciplines, to explore four specific aspects of the mind - subjectivity, intentionality, consciousness and agency. The course ran in the first quarter of 2015 and again in November 2015. Data from both runs is included in this paper. Discussion Teacher presence Teacher presence significantly influences learner motivation in online and distance education (Anderson & Dron, 2011; Ross et al., 2014). Effective online instruction requires teachers to be actively involved in their students’ learning (Dixson, 2010). In traditional online and Author accepted manuscript April 2016 9 distance learning, a number of models might be deployed to enable the presence of a teacher in synchronous and asynchronous modes such as virtual small tutor groups, virtual tutorials, call centres and discussion forums with teachers signalling presence through leaving comments on discussions, interacting with groups of students virtually, posting feedback as replies to student input and giving assessment feedback. In MOOCs, learners do not expect direct interaction with the educator due to the large numbers of enrolled learners and the informal enrollment. This lack of direct supported instruction is commonly cited as a limitation of MOOCs as high quality effective learning spaces and is seen as a contributor to non completion rates and drop-outs. A study from the University of Edinburgh’s Elearning & Digital Cultures MOOC indicated that many participants felt adrift and confused as to where the teaching was (Knox, 2014) and such negative student feedback has been reported in a study of a number of connectivist MOOCs (Kop et al., 2011) However various strategies have been deployed to enhance the sense of teacher presence (Hew, 2014) or at least provide proxies for the teacher in MOOCs. One of the earliest strategies has been establishing the presence of the educator through the use of video lectures. MOOC videos usually take the form of short “talking head” lectures, screencasting, dynamic drawing, animations or a combination of styles. The effect of these more intimate ‘tutorial’ style videos is to bring the instructor to life in the eyes of the learner rather than the learner being a bystander. Many MOOC video styles exist and learners react in different ways to the different styles, with indications that authentic, dynamic and short lectures engage learners so that they keep watching (Guo et al, 2014). Our experience with Medicine and the Arts and What is a mind? supports this research that short videos proved engaging, with learners responding to the videos by leaving comments focussing on the personality and style of lecturing of the lead educator with whom they felt an individual connection as illustrated in the comments below: Very grateful for access to such a terrific course. Great to have Prof Solms' direct, enthusiastic robust/muscular teaching style and nuanced, humane approach. I did always feel as though he was addressing each of us individually. Learner A Author accepted manuscript April 2016 10 I don't think it would be possible for a better teacher/lecturer/enthusiast than Dr Mark! He has made the course absolutely fascinating, and his sense of humour has enhanced his teaching. Learner B Some learners responded directly to the educator as if they were having a conversation as in this case: you express mystification as to why you always think that the artist of the Blombos cave etching was female. Well the etched lines quite evidently spell out an XX pattern. Can you have repressed the obvious Freudian allusion?! Learner C The presence of the educator is also signalled in less direct ways in the MOOC platform interface, so that small picture icons of the educators appear whenever they leave a comment as well as a role label denoting their status, while the quiz feedback text is written as if it were from the educator. This deliberate strategy of writing instructional text from the point of view of the educators using the first person “we”, use of their first names and pictures are platform design features to help learners feel a sense of connection with the educators. With the Medicine & the Arts MOOC, this was particularly important as this MOOC comprised some 18 different presenters with the two lead educators in the position of convening, hosting and synthesising the learning. Many MOOCs deploy course mentors/facilitators to interact with learners to provide more expert facilitation. In both of our courses, senior students at Master’s and PhD level with subject matter knowledge were employed to provide facilitation in the course discussion spaces and in the case of What is a mind? to select participant questions for the lead educator to respond to. Learners appeared to appreciate the presence of the course mentors, sometimes calling on them directly in course discussion spaces, and using the platform social-media conventions to “follow” comments made by the mentors. Many learners recognised the limited engagement of educators but seem to be aware of the MOOC format constraints: So appreciative that all these highly busy people have given the time to share their experiences with us - thank you. Learner D Another example of enabling instructor presence at scale was deployed in the ‘What is a Mind?’ course where each week learners were invited to pose questions and queries in an Author accepted manuscript April 2016 11 ‘Ask Mark’ forum. The pedagogical purpose of this activity was to enable learners to pose any misconceptions or questions that emerged as they worked through the week’s content and activities with a commitment that the instructor would respond to a selection of questions by recording answers as a video, which was posted on the course website at the beginning of the following week. This activity elicited considerable engagement, first on the part of the learners who not only submitted questions but also commented on and augmented fellow learners’ questions and in some cases answering some of the questions. Learners also responded positively to the videos recorded by the educator with some rating this one of the the best feature of the course and in relation to other MOOCs: The triumphant feature, as so many of your students remarked, was the "Ask Mark" videos. The process of filtering and consolidating the questions for the instructor to reply to was far superior to various open line exchanges employed by other MOOC's. Learner E Totally brilliant answers, it is especially great to be on a course where the participants questions are really listened to. Learner F Although the ‘Ask Mark’ strategy was deliberately designed to respect the limited time the educator had, this limitation was turned into a positive feature in that the role of others in posing questions, in selecting the question, filming, editing and uploading videos allowed a rich interaction with learners and increased engagement. Regular course communication between learners and the instructor via digital communication channels enables a virtual link between educators and students in online learning. Dixson (2011) in a study to ascertain what engaged students in online courses, found that while there was no particular pedagogical activity that engaged students the different ways of communicating between student and instructors and between students during the course led to higher levels of student engagement. In a MOOC, channels of communication are not individualised but weekly email announcements from the educator summing up course developments and looking ahead to what was coming up proved to be motivating. In both courses these emails are clearly motivating students to return to the course on a weekly basis (Figure 2). Author accepted manuscript April 2016 12 Figure 2: Heatmap for the ‘What is a Mind?’ course showing patterns in steps completed. An e-mail announcement is sent at the start of every week. One can observe the impact of these weekly emails on learner behaviour. Establishing teacher presence in MOOCs in the absence of direct instruction involves finding innovative and practical ways of making the most of an academic’s limited time, amplifying what interaction is possible and providing a number of proxies. This is “light-touch” teaching at scale which we speculate might provide learners with a sense of confidence even in the absence of direct contact from the lead educator. Social learning While there is less teacher involvement in MOOCs when compared to traditional university courses, the teacher involvement is far greater when compared to say educational books or Author accepted manuscript April 2016 13 television. It is this “in-between” publishing space that MOOCs are helping explore how large numbers of people can become engaged through online forms of social interaction. We use the term ‘social learning’ here to distinguish this from design choices that are centred on the educator and peers. The FutureLearn platform reflects a social constructivist approach to learning and its design, which is based on Laurillard’s conversational framework (2002), encourages a learning approach based on conversations between participants (Ferguson and Sharples, 2014). On MOOC platforms, there are opportunities to use social media, which we also include here as it is used to engage with others outside the course. We focus on how learners have engaged socially, seen either as general behaviour patterns within the platform or engagement outside of the platform. The implicit premise is that learners and learning benefit from engaging with others through conversations and interactions. Social learning is not focusing on any new or specific strategies in online learning design, but we highlight some manifestations in a MOOC format that merits analysis. These may lead to better insights for designers given the constraints of the MOOC format. When a learner watches a video, reads an article or responds to an activity prompt they can often post a comment in that “step” or respond to other learners’ comments thus initiating a conversation. Of all the learners on the course who visit one or more steps, about 40% post a comment at least once. Figure 3 shows the total number of comments made by learners over the 42 day duration of the ‘What is a Mind?” course. The first few days had the highest number of comments, many of which were learners introducing themselves. The different patterns in each bar are used to indicate the week of the step in which the comment was posted. We interpret that some people worked through the course quickly and had started engaging with material for the second week on the first day while some discussions continued for longer than 7 days. Overall the pattern of engagement is strongly driven by the course calendar as was depicted in Figure 2. Monday has the largest number of postings, as people receive e-mail reminders indicating the course design is helping people ‘learn together’. Author accepted manuscript April 2016 14 Figure 3: Comments posted per day for the duration of the course, with different patterns representing comments for the distinct weeks into which the course is organised. Visits and comments show strong patterns in behaviours, preferences and ambivalence to discussions. Post-course surveys (see Figure 4) indicates the mode with the highest overall positive rating is video, followed by resources such as readings and external links, then quizzes while there is greater ambivalence towards what peers have contributed. Based on the different course runs this suggests there are typically 5% to 15% of learners who would prefer not to read comments by others or avoid contributing to discussions. Further, a high proportion of people do not respond to this question, likely because they do not engage with comments at all or are indifferent. This is substantiated by written survey remarks where people have expressed their dislikes and mistrust of peer comments. This is a useful observation for learning designers promoting social learning strategies. Author accepted manuscript April 2016 15 Figure 4: The responses from the post-course survey for the question ’How did you feel about learning on FutureLearn’. Nonetheless we found clear indications that some learners were exhibiting high levels of generosity and expending energy to find and post resources for other learners in the comment spaces, for which other learners voiced their appreciation: I've really enjoyed hearing about such a variety of topics but have actually found the comments made by other participants to be just as, if not more at time, interesting. Learner G While the number of learners fully participating as social learners was around 40% (where a learner has posted a comment at least once), the presence of this core group had a significant impact on the learning environment, enabling vicarious learning by others. The presence and contributions of fellow learners can encourage engagement and broaden perspectives for learners. Courses cannot go into depth on all aspects of a topic and intentionally invite learners to contribute their experiences and perspectives: I realize this course was tentative in juxta-positioning two extreme subjects ie hard and soft science. The lack of arts representation may reflect the lack of support for Author accepted manuscript April 2016 16 arts in medicine as it were. However the wealth of comment indicates there are many intelligent and insightful people who will build on the foundation this course has provided and take it further. Learner H Another thanked a fellow learner for filling a perceived gap in the course: Thank you. I had been thinking that the arts were being lost in this week's talks. But by bringing in Pinter and the Aborigines' 'Songlines' you have brought us back to the course title, 'Medicine and the Arts'. Learner I Learners also engaged socially outside of the FutureLearn platform in a number of ways: sometimes in response to prompts from the course designers to share artefacts to social media websites, while other times learners share without prompting. In ‘What is a Mind?’ learners were invited to post images of the mind that resonated with their understandings to Padlet (an online bulletin board). There are just over 50 individual images, with nearly three times as many references to these in the discussion comments showing strong engagement with this activity. While we can see social learning happening in the course as a whole, the experiences of individual learners, from their perspective, can be quite different. Some learners remark how they might get overwhelmed by comments or avoid reading them completely. For many learners it is not feasible to engage with all the comments or to continue to return to the forum. Thus generally learners are getting a partial view of the social learning taking place. When we attempted to analyse the data more from an individual's’ point of view, slightly different perspectives emerged. For example, just over 10% of learners started in the first week and were still visiting the course six weeks later. It was much more likely that learners had a much shorter time to engage on the course. Of the total learners that visited at least one step, 42% only spent one day visiting the course. As Figure 2 also shows, some learners had completed the course on the first day. For learning designers when designing for social learning, strategies for social engagement over a sustained period of time may be difficult to maintain as learners’ time on the course appears to be short and concentrated. Author accepted manuscript April 2016 17 Peer learning Peer learning and peer teaching, where MOOC participants interact with each other around core course content, teach others and review each other's’ assignments, has been heralded as a significant innovation in online learning allowing for personal interaction at scale. Aside from the affective benefits of active participation, open enrollment results in large numbers of participants, including people with high levels of expertise. This may include other educators but also professionals, practitioners, students and in the case of the Medicine and the Arts MOOC many patients. MOOC audiences are generally well-educated (for example, 83% of participants on the Medicine and the Arts course went to university or college), and attract highly qualified or expert individuals who may step into a peer educator role. In the second run of Medicine and the Arts during the week on the topic of heart transplants a participant posted a comment wondering whether scientists would be able to create hearts from stem cells in the near future. A few posts later, another participant responded from a position of considerable expertise: I was part of Prof Christiaan Barnard's first and second "Twin/Double" heart transplant teams... . Given that experience - of what is possible - I believe it will be a stem-cell heart will be achieved; in a while! Learner J One of the lead educators on the Medicine and the Arts course was particularly struck by the ‘democratic’ nature of the space - without many of the clues about status from face-to-face encounters (such as age, gender, appearance, language and so on), the virtual shared classroom immediately broke down the barriers of authority: MOOCs are a ...sort of levelling platform, ... an undergraduate and a postgraduate and a senior experienced person are all on the same level in a MOOC. So everyone’s comments ... carry equal weight. (Interview with lead educator) This dynamic was both an enabler - allowing people who might normally be silenced to participate and a constraint on learning. On the enabling side, the MOOC learners accept the limitations of engagement by the educators and often take on ‘teacherly’ roles. Part of the reward of engaging in learning in a MOOC can be the flattened teacher-student relationship. Author accepted manuscript April 2016 18 There is still an expectation that the lectures and content will provide new knowledge and learning, but there seems to be more shared responsibility for learning. In discussion initiated by the educators about the definitions of Medical Humanities, a medical doctor posted a well- formulated response with stimulating questions for other participants - which received the highest number of ‘likes’ on the course - a clear signal of other learners’ acknowledgement of the intellectual credibility of a peer learner. Many are interpreting "medical humanities" as a way of making modern medicine more humane, because it is allegedly technology-bound, hyperspecialised and distant from the actual people and their problems. As a junior doctor I can say that this should be only part of the field's aim, also because - let's not forget - technology, drugs and specialisation are a great advance to medicine. Learner K The comments sections were well utilised and those participating demonstrated appreciation for the inputs of other learners - acknowledging them directly or actively soliciting comments from peers. I am still confused .. I can't say that Animals have no brain by these examples. What to do...? Learner L On the other hand, in some instances, there was some frustration expressed about the differing levels of engagement, especially on the part of some educators who compare the MOOC unfavourably to mainstream postgraduate classroom teaching. For instance, one of the guest presenters on Medicine and the Arts was concerned about what a “thin little slice, that the MOOC cuts, you know, across a domain of topics” (interview with educator), with few readings compared to a university credit-bearing course. We observed that often discussions remained at a superficial level, with students bringing in personal anecdotes to illustrate a point or departing from the topic. Yet seemingly superficial conversations have a value in providing affective and emotional support and given the varied backgrounds, knowledge, language levels, ages of participants, this is not surprising to find. We saw examples of learners making personal connections and arranging to take discussions to other private spaces. https://www.futurelearn.com/profiles/716249 Author accepted manuscript April 2016 19 Most major MOOC platforms offer a “peer review” feature for assignments where student assignments are read and commented on by a fellow learner rather than the course academic or teaching assistants, as might be the case in a formal online course. Learners are scaffolded to give peer feedback through the provision of a rubric. The pedagogical value of peer review lies in both the opportunity to read and learn from reviewing another’s work as well as getting feedback from someone else. Peer review assignments were used in both courses and MOOC participants could see who has reviewed their work, in order to build up personal exchanges. In addition to the visibility of the peer reviewer’s identity, we designed the peer review activities so as to allow broader public peer review by inviting learners to share their assignments with others taking the course as well as with their designated peer reviewer. This was a deliberate design strategy to give learners who might have received an indifferent or sparse review the opportunity to get feedback from a broader group of learners. We noticed the lack of anonymity encouraged constructive engagement and direct responses between learners.The majority of the comments were remarkably positive. The tone of much of the interaction was a mixture of teacher-style feedback, along with a cohort consciousness of a shared experience: I was at university 30 years ago and no such thing as peer assessment then. I was a little apprehensive about this for this assignment because we presumably come from very different starting points… . But reading the comments on this discussion virtually everybody who commented found the experience of marking and of receiving feedback to be helpful to their understanding of the topic. Learner M This was a very useful exercise in reviewing other people’s work. So many different creative points of view to take on board as well as sharpening the critical skills needed to evaluate the texts and content. Learner N While peer review assignment offers opportunities for learners to receive constructive feedback we have found limitations of the process, especially in situations where participants have very different backgrounds, prior knowledge and language skills. https://www.futurelearn.com/profiles/1136799 Author accepted manuscript April 2016 20 Alfonso thank you for reviewing my assignment. Could I ask that you perhaps write it in English as I do not understand Spanish (?). Learner O Some learners expressed disappointment with their peer review feedback although this was sometimes mitigated through the social learning features: I must state that the reviewer of my assignment was totally distracted by his own verbosity and seems to have been looking for some extra space to write another essay of his own. Not once was anything I wrote even referred to. Learner P I don’t agree with this kind of assignment, to give your opinion on other adult´s work.The interpretation is something personal and should be respected.I do not feel well doing something that I am not sure enough. I worked uncomfortably not convinced on what I was doing and I apologize for criticising the work of my course mates because I am not the educator to do such a thing. Learner Q Such shared experiences were reflected on by other students, and a variety of responses making sense of this new learning space were discussed: MOOCs such as this attract many thousands of students and much of the feedback has to come from fellow students. In my experience the most valuable learning from FutureLearn courses comes from student to student interaction. Inevitably, some responses will cause offence and distress. It is our duty to try to prevent this in our interactions. Learner R It is a given that the composition of a virtual classroom in a MOOC course is fundamentally different to a standard credit-bearing university class; there is enormous diversity - age, nationality, language, life and work experience, culture, disciplinary background - and this represents both the potential and limitation of the mode. If well-conceived, being able to harness the diversity through tools which allow for peer teaching and peer learning is one of the most powerful potentials of MOOCs not possible in formal online learning. However we found both positive and negative attitudes to peer reviews suggesting that careful design of the activity is required to engage learners, accept that some reviews will be sub-standard and respect learners’ time in submitting and reviewing the assignments of others through pacing and selective use of the peer review mode. Author accepted manuscript April 2016 21 Our review of the platform analytics and in particular the attrition rates that track when participants did not proceed to the next stage of the course (Figure 5), suggests that the regular pattern of seeming non-engagement is attributable largely to people’s competing demands on time rather than any particular design flaw in the course. This interpretation is backed up by the importance of the weekly course emails that saw spikes of activity and the end of week step that seemingly marked a closure for students even thought they could have proceeded through the MOOC. Figure 5: The proportion of learners continuing to the next step in the ‘What is a Mind?’ course. Overall the proportion continuing is above 95%. For the first step (1.01) about 10% did not continue Conclusion Learner engagement is a challenge to design for in MOOCs. In conventional courses and assessments it is often easier to provide evidence for engagement and learning as it is much clearer how people are responding and what they are expected to achieve. In MOOCs people are not expected to engage if they choose not to; they may simply be curious or be very selective in what they take from the course. Designing for engagement in MOOCs not only Author accepted manuscript April 2016 22 acknowledges students’ differing motivations but also leverages the massiveness and diversity of students attracted to promote valued and diverse forms of engagement. In observing learner interactions and then interpreting these interactions through the lenses of three well-known pedagogical concepts - teacher presence, social learning and peer learning - we are able to probe for richer meaning than analytics are able to provide; this helps inform the design decisions to enhance or amplify engagement given the affordances of the MOOC format. Most visible in our study are a comparatively small core group of learners actively and deeply participated in all these sharing practices; their behaviour enabled ‘vicarious’ learning and enhanced the entire learning experience for the broader community. We also considered what might be inhibiting further such participation. The vast majority of signups have low participation and we identified some characteristics of these people, such as never returning. This may be a feature of a cohort based learning design model at scale. Further research may involve creating instruments to delineate specific features of teacher presence, social learning and peer learning to enable comparison across different courses and surveying learners to gauge perceptions and practices. The online curriculum landscape is presently shifting and coalescing; it has not yet settled into rigid structures which determine and shape learners’ virtual practices. This study makes a contribution to showing how learning designers can exploit the opportunities which presently exist to develop different strategies for supporting learner engagement in innovative ways. Notes on contributors Sukaina Walji is a project manager and learning designer based at the Centre for Innovation in Learning and Teaching (CILT) at the University of Cape Town. Andrew Deacon is a learning designer in the Centre for Innovation in Learning and Teaching (CILT) at the University of Cape Town. Janet Small is a course developer based at the Centre for Innovation in Learning and Teaching (CILT) at the University of Cape Town. Author accepted manuscript April 2016 23 Laura Czerniewicz is the director of the Centre for Innovation in Learning and Teaching (CILT), a unit in the Centre for Higher Education Development at the University of Cape Town. Author accepted manuscript April 2016 24 References Anderson, T., & Dron, J. (2011). Three Generations of Distance Education Pedagogy. International Review of Research in Open and Distance Learning, 12(3). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/890/1826 Czerniewicz, L., Deacon, A., Small, J., & Walji, S. (2014). Developing world MOOCs: A curriculum view of the MOOC landscape. Journal of Global Literacies, Technologies, and Emerging Pedagogies, 2(3). Retreived from http://www.jogltep.com/index.php/JOGLTEP/article/view/23/10 de Freitas, S. I., Morgan, J. and Gibson, D. (2015). Will MOOCs transform learning and teaching in higher education? Engagement and course retention in online learning provision. British Journal of Educational Technology, 46, 455–471. doi: 10.1111/bjet.12268 Dixson, M. (2010). Creating effective student engagement in online courses: What do students find engaging? Journal of the Scholarship of Teaching and Learning, 10(2), 1–13. Retrieved from http://josotl.indiana.edu/ Ferguson, R., & Sharples, M. (2014). Innovative pedagogy at massive scale: teaching and learning in MOOCs. In Rensing, C., de Freitas, S., Ley, T., Munoz-Merino, P.J. (Eds.), Open learning and teaching in educational communities (pp. 98–111). Graz, Austria. Springer International Publishing. doi:10.1007/978-3-319-11200-8_8 Ferguson, R., Sharples, M., & Beale, R. (n.d.). Enabling social learning on Futurelearn. Retrieved from https://ugc-partners.futurelearn.com/wp-content/uploads/2013/06/2.- Social-Learning-in-MOOCs.pdf Guo, P., Kim, J., & Rubin, R. (2014). How video production affects student engagement: an empirical study of MOOC videos. In Proceedings of the first ACM conference on Learning @ scale conference (L@S '14). ACM, New York, NY, USA, 41-50. http://doi.acm.org/10.1145/2556325.2566239 http://www.irrodl.org/index.php/irrodl/article/view/890/1826 http://www.irrodl.org/index.php/irrodl/article/view/890/1826 http://www.jogltep.com/index.php/JOGLTEP/article/view/23/10 http://www.jogltep.com/index.php/JOGLTEP/article/view/23/10 http://www.jogltep.com/index.php/JOGLTEP/article/view/23/10 http://www.jogltep.com/index.php/JOGLTEP/article/view/23/10 http://josotl.indiana.edu/ http://josotl.indiana.edu/ https://ugc-partners.futurelearn.com/wp-content/uploads/2013/06/2.-Social-Learning-in-MOOCs.pdf https://ugc-partners.futurelearn.com/wp-content/uploads/2013/06/2.-Social-Learning-in-MOOCs.pdf Author accepted manuscript April 2016 25 Hew, K. (2014) Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCS. British Journal of Educational Technology, 47, 320–341. http://dx.doi.org/10.1111/bjet.12235http://dx.doi.org/10.1111/bjet.12235 Ho, A,. Chuang, I., Reich, J.,Coleman, C.,Whitehill, J., and Northcutt, C., Williams, J., Hansen, J., and Lopez, G., Petersen, R. (2015). HarvardX and MITx: Two Years of Open Online Courses Fall 2012-Summer 2014. http://dx.doi.org/10.2139/ssrn.2586847 Jona, K & Naidu, S. (2014). MOOCs: emerging research. Distance Education, 35(2), 141– 144. doi:10.1080/01587919.2014.928970 Kahu, E. R. (2013). Framing student engagement in higher education. Studies In Higher Education, 38, 758-773. doi:10.1080/03075079.2011.598505 Knox, J. (2014). Digital culture clash: “massive” education in the E-learning and Digital Cultures MOOC, Distance Education, 35(2), 164-177, doi: 10.1080/01587919.2014.91770 Kop, R., Fournier, H., & Mak, J. (2011). A pedagogy of abundance or a pedagogy to support human beings? Participant support on massive open online courses. The International Review Of Research In Open And Distributed Learning, 12(7), 74-93. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1041/2025 Laurillard, D. (2002). Rethinking University Teaching. A conversational framework for the effective use of learning technologies. London: Routledge. Liyanagunawardena, T. R., Lundqvist, K. Ø. and Williams, S. A. (2015). Who are with us: MOOC learners on a FutureLearn course. British Journal of Educational Technology, 46, 557–569. doi: 10.1111/bjet.12261 Maxwell, J. (2005). Qualitative Research Design: An interactive approach (2nd ed.). Thousand Oaks, CA: Sage Robinson, C., & Hullinger, H. (2008). New benchmarks in higher education: Student engagement in online learning. Journal of Education for Business, 84, 101–108. doi:10.3200/JOEB.84.2.101-109 http://dx.doi.org/10.1111/bjet.12235 http://dx.doi.org/10.1111/bjet.12235 http://dx.doi.org/10.1111/bjet.12235 http://dx.doi.org/10.1111/bjet.12235 http://www.irrodl.org/index.php/irrodl/article/view/1041/2025 http://www.irrodl.org/index.php/irrodl/article/view/1041/2025 http://www.irrodl.org/index.php/irrodl/article/view/1041/2025 Author accepted manuscript April 2016 26 Ross, J., Sinclair, C., Knox, J., Bayne, S., & Macleod, H. (2014). Teacher experiences and academic identity: The missing components of MOOC pedagogy [MOOC Special issue]. Journal of Online Learning and Teaching, 10(1), 57–69. Retrieved from http://jolt.merlot.org/vol10no1/ross_0314.pdf Stewart, B. (2013). Massiveness+openness=new literacies of participation? MERLOT Journal of Online Learning and Teaching, 9(2), 228-238. Retrieved from http://jolt.merlot.org/vol9no2/stewart_bonnie_0613.htm Trowler, V. (2010). Student engagement literature review. York: Higher Education Academy. Retrieved from https://www.heacademy.ac.uk/sites/default/files/studentengagementliteraturereview_1 .pdf Veletsianos, G., Collier, A. and Schneider, E. (2015). Digging deeper into learners' experiences in MOOCs: Participation in social networks outside of MOOCs, notetaking and contexts surrounding content consumption. British Journal of Educational Technology, 46, 570–587. doi: 10.1111/bjet.12297 Yuan, L., & Powell, S. (2013). MOOCs and Open Education: Implications for Higher Education - A white paper, JISC CETIS (Centre for Educational Technology & Interoperability Standards). Retrieved from http://publications.cetis.ac.uk/2013/667 http://jolt.merlot.org/vol10no1/ross_0314.pdf http://jolt.merlot.org/vol10no1/ross_0314.pdf http://jolt.merlot.org/vol10no1/ross_0314.pdf http://jolt.merlot.org/vol10no1/ross_0314.pdf http://jolt.merlot.org/vol9no2/stewart_bonnie_0613.htm http://jolt.merlot.org/vol9no2/stewart_bonnie_0613.htm http://jolt.merlot.org/vol9no2/stewart_bonnie_0613.htm http://jolt.merlot.org/vol9no2/stewart_bonnie_0613.htm https://www.heacademy.ac.uk/sites/default/files/studentengagementliteraturereview_1.pdf https://www.heacademy.ac.uk/sites/default/files/studentengagementliteraturereview_1.pdf http://publications.cetis.ac.uk/2013/667 Author accepted manuscript April 2016 27 Appendix A According to the licensing conditions of the FutureLearn platform, the authors of the comments quoted in the text are listed below: Learner A Sue Davies Learner G Liffey Speller Learner M Robert Wilson Learner B Althea Stevens Learner H Sylvia Thompson Learner N Terry Chinn Learner C Neil McLellan Learner I Judith Thomas Learner O Alda Smith Learner D Michael Pasternak Learner J Jonathan Smith Learner P Donal Buckley Learner E Neil Carson Learner K Andrea Mazzella Learner Q Maria de las Mercedes Quintana Learner F Lisa Byford Learner L Padma Kala Learner R David Lillystone Figure captions Figure 1: Higher education course provision landscape Figure 2: Heatmap for the ‘What is a Mind?’ course showing patterns in steps completed. An e-mail announcement is sent at the start of every week. One can observe the impact of these weekly emails on learner behaviour. Figure 3: Comments posted per day for the duration of the course, with patterns representing comments for the distinct weeks into which the course is organised. Figure 4: The responses from the post-course survey for the question ’How did you feel about learning on FutureLearn’. Figure 5: The proportion of learners continuing to the next step in the ‘What is a Mind?’ course. Overall the proportion continuing is above 95%. For the first step (1.01) about 10% did not continue. i https://www.futurelearn.com/courses/medicine-and-the-arts ii https://futurelearn.com/courses/what-is-a-mind https://www.futurelearn.com/courses/medicine-and-the-arts https://futurelearn.com/courses/what-is-a-mind