A Case Study In Distance Education A Case Study In Distance Education The Information Resource Management Program: A Case Study In Distance Education By: Steven Cohen & William Eimicke, Columbia University Mark Kamlet & Robert Pearson, Carnegie Mellon University February, 1998 Introduction The speed and scope of change in information technology has created special challenges as well as new opportunities for us all. As a society, we must understand and come to terms with the social, political and economic effects of the information age. As managers and public policy practitioners, we must recognize the need for improving our skills and knowledge in this area. Fortunately, some of the same information technologies that pose these challenges are also improving the way in which educational programming can be delivered. In 1996, the H. John Heinz School of Public Policy and Management at Carnegie Mellon University and the School of International and Public Affairs at Columbia University joined forces to launch an innovative distance learning program in Information Resource Management (IRM). The program is innovative in two respects. First, it uses new and some not so new technology to meet the educational needs of those who cannot easily attend conventional degree programs. Second, the program is a collaboration between two graduate schools committed to exploring the new learning opportunities created by technology advancements. The IRM program provides a one-year, six-course curriculum designed to strengthen managerial and analytical skills and improve understanding of the role of information technology in work, the home and society at large. The initial class offerings include: Analysis and Design of Information Systems; Information Technology and Social Accountability; Database Management; Geographic Information Systems; Decision Support Systems; and Telecommunications Management. All courses begin and end with face-to-face instruction in New York, but the bulk of course work is conducted through the use of videotapes and the World Wide Web. As the program completes its second year, we thought it potentially useful to develop a case study documenting our experiences in creating and running this new program. The study seeks to answer the file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (1 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education following questions. What could the literature on distance learning tell us about the best way to structure the program? How did we begin? How does the program differ from traditional graduate courses? What have been the experiences and perspectives of those who set off on this journey into the relative unknown - the students, the faculty and our respective administrations? What challenges emerged and how did we try to overcome them? How much did the program cost to develop and to operate and what are the universities' return on these investments? And what does the future hold for our program and others like it? The Literature of Distance Learning Nearly all of the literature on distance education begins with a definition. While no two definitions are exactly alike, most define distance education as an approach to learning in which: ● teacher and learner are separated by space and/or time; · the interaction between teacher and learner takes place via a technology link; and · students are evaluated by an educational organization. From these three principal characteristics flow an incredible variety of distance education programs. They vary with respect to whether: the interaction between teacher and learner is conducted in real time or asynchronously; is principally one-way or two-way; relies on video, audio, and/ or text; or involves one instructor and many students or one or more instructors and few students. Distance education programs also differ in the ways they combine different delivery modes, including elements of conventional face-to-face instruction. Distance education programs target a variety of audiences including K-12, baccalaureate, and graduate levels. The curricular content also ranges widely. Graduate level programs are generally in business administration, information systems, and engineering. Overall objectives range from simply providing information to creating an environment for mastering complex skills. And the technical quality of interaction can differ greatly with respect to the speed and clarity of video or audio transmission. This variety makes it difficult to study and compare distance education programs. It is equally difficult to summarize the literature on the subject. Our own modest attempt at reviewing the literature suggests that: · the literature on distance education is vast and growing quickly, as is the development of distance education programs; · a large proportion of this literature is written from the vantage point of those who have a stake in its success (i.e., tends towards promotion); · few rigorous evaluations have been conducted (i.e., selection biases are not accounted for in comparing file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (2 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education the outcomes - often measures of student satisfaction - between conventional classroom instruction and distance education); · the literature has begun to move beyond such comparisons by asking under what conditions new instructional technologies contribute positively to learning; · cost effectiveness or cost savings- although often claimed - have not been systematically documented; · the results of empirical studies tend to show distance education outcomes as good as, if not better than, those achieved in conventional classrooms; · distance education programs are the only instructional alternative for substantial groups in the population or provide resources to instructors that they would not otherwise have; · while self-selection biases confound statistical estimates of the effects of distance education programs and may explain the positive results documented in many observational studies, this same condition means that some students (e.g., self-directed) do very well in this format and that recruitment into such programs should take these characteristics into account; and · communication technologies used in programs tend to minimize non-verbal social cues (e.g., gender, age) and thus promote greater, and more egalitarian, interaction among participants. It is also possible for the technology to generate too much interaction among students, which can result in attrition or disengagement. A number of case studies suggest that 25 students is optimal for classes that rely on computer-mediated communication. Case Study: The Information Resource Management Program The Information Resources Management Program (IRM) began as a concept paper developed by the Heinz School during the summer of 1995. The original idea was to offer in a distance learning (DL) format, using a combination of technologies, six graduate level courses in Information Policy and Management which were, and still are, being taught at the Heinz School to on-site students. The principles and objectives that guided our choice of technologies were to: · use robust, relatively inexpensive, and widespread technologies (e.g., videotapes, World Wide Web); · use technologies to promote thoughtful interaction and cooperative learning among students as much as between instructors and students; · plan for continuous development of course materials as instructional and telecommunication technologies evolve; file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (3 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education · use a combination of technologies in which face-to-face instruction still plays a role; and · create a learning experience that meets or exceeds the expectations that students bring to the program because of the reputations of our respective institutions and/or because of the level of tuition we charge. The two target markets initially identified were Washington, D.C. and New York City. Heinz's first choice for a New York City partner was the School of International and Public Affairs (SIPA) at Columbia University. This preference became stronger when an initial inquiry to Georgetown University in Washington, D.C. evidenced little faculty interest in the partnership and constraints in the use of already fully utilized facilities. Heinz chose SIPA because of the school's top quality public policy and management program which was comparable and complementary to Heinz's Master of Public Management degree. SIPA's New York City location was a major advantage as a large market, as was the school's association with the internationally recognized Columbia University. Finally, Heinz anticipated that the IRM courses might serve as an attractive concentration for the master's degrees at both schools. The joint program seemed to make sense to both schools because they believed there was a substantial market of working adults interested in master's level courses of this type who could not easily access traditional, full-time residential degree programs. Second, like many other public policy schools, SIPA saw the need and demand to provide cutting edge courses in information policy and management but lacked the in-house capacity and/or financial resources to offer these courses itself. SIPA had requests from its own students for such courses and also sensed a potential strong demand from mid-career students working in all three sectors throughout the New York Metropolitan region. The Heinz School had over a decade of experience in teaching information policy and management, has one of the strongest faculties in the world in this field and had the equipment and expertise readily available at Carnegie Mellon to transfer its traditional courses into a Distance Learning format. Heinz could access the University's state of the art television theater class rooms and a technical staff capable of producing commercial quality video tapes. Heinz's faculty were also already making highly innovative use of the computer and the World Wide Web as a teaching and communications tool. Heinz's IRM team did extensive research on distance learning experiments around the country and became convinced that the format would work well for these technology-based courses with mid-career students who were already working in the information resources area. New York City's history as place of innovation and change suggested to the Heinz team that the mid-career market there would respond well to this flexible and hands-on approach to graduate education. The Heinz project team included Robert W. Pearson, director of the School's executive master's degree program and Mark S. Kamlet, Dean of the School. The SIPA team consisted of William B. Eimicke, director of SIPA's Public Management Training Center, Steven Cohen, SIPA's Associate Dean and Director of the MPA program, and John Ruggie, SIPA's Dean. The direct involvement of Deans Kamlet and Ruggie in the early stages of program definition and in the various internal and external approval file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (4 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education processes proved to be an important ingredient of the program's ultimate success. The administrative and financial hurdles were many and the project could have been stopped numerous times along the development path were it not for the bureaucratic skill and "halo" effect of the two deans' direct participation in the planning stage. Cohen managed the administrative, financial and curriculum issues throughout the complex Columbia University approval processes. Pearson and Eimicke, who became the program coordinators, put together the budget, the marketing plan, kept the faculty and technical staff enthusiastic and on board, and managed the logistics of offering six courses from two schools 800 miles apart. The key administrative issues that arose early on included student status, administrative approvals, advising and the course schedule. 1. Whose Students, Which Transcript? Two different types of students were targeted initially: regular SIPA public administration degree students for whom the six course certificate would serve as their master's degree concentration; and working professionals seeking career enhancement skills who might later consider a master's degree at either Carnegie Mellon or Columbia University. For the second group, their student status and privileges during the certificate period required clarification. For matriculated SIPA students, their status was clear: they would be SIPA students and receive SIPA transcripts and credits for the IRM courses. The part-time students, it was agreed, would be Carnegie Mellon students and receive Carnegie Mellon transcripts and credits for the IRM courses. This approach was easiest and most logical, since the faculty and courses were already a regular component of the Heinz school curriculum. To help market the program, the schools agreed that all credits earned through the program would be applied toward a master's degree at either school and would fulfill the concentration requirements for that degree. Successful completion of the certificate qualified students for admission to the Heinz School's Master of Public Management Program. Admission to SIPA's master's degree required a separate application, but SIPA would indicate in writing that proven success in the IRM program represented extremely strong credentials for acceptance into the master's program. 2. Columbia University Approvals Required- There were minor administrative issues for Carnegie Mellon in offering existing courses with their own faculty but in a distance learning format. The administrative issues faced on the Columbia University side of the partnership were a bit more complicated. SIPA had to pre-qualify the Carnegie Mellon faculty and list them as Adjunct Professors at SIPA. It was agreed that they would be listed at the same level as their rank at Carnegie Mellon: assistant, associate or full professor. SIPA's Committee on Instruction also had to approve the IRM courses for them to be listed as credit-bearing SIPA courses. Cohen and Ruggie informed and managed this process so that no delays or controversies developed. 3. Student Advising and Accommodations- Bill Eimicke, already an advisor to SIPA's regular MPA students, agreed to assist and advise IRM students through SIPA's Public Management Training Center. Located in the same building as the class rooms and computer laboratory that the IRM students would use, Eimicke's office would be convenient for the mid-career students and his fifteen years as a public file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (5 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education sector manager made him a good choice to relate to these older, practitioner-oriented students. Eimicke would also rule on possible waivers of SIPA's internship and practicum requirements for IRM students continuing on for a master's degree in New York. IRM students would be given Columbia University identification cards offering full access to SIPA's computer labs, libraries throughout the university and other resources and amenities, such as the university athletic facilities, pools and tennis courts. SIPA class rooms used for the first and last in- person course sessions as well as other seminar rooms would be available to IRM students in the evenings and weekends for study sessions and group projects. Meeting space could also be arranged during the busy day periods by appointment through Eimicke's office. 4. Course Schedule- The initial plan of three courses for each of two consecutive semesters was modified to two courses a semester for three consecutive semesters: Fall, Spring and Summer. Informal surveys of current and prospective students indicated that completion of the certificate within twelve months was desirable but three courses at one time would be too heavy a burden for most older students working full-time, managing a home and family, and several or more years away from their last higher education course work. Finances were also a major area of concern from the beginning of the partnership. The schools ultimately agreed to charge $2,200 per course initially, or about the same amount that the Heinz School was currently charging for its mid-career, executive master's program course offerings. No financial assistance was available from the partners, although it was hoped that the majority of students would be eligible for at least partial reimbursement from their employers. Heinz agreed to help students apply for available government loan programs. Heinz invested substantial sums in the development of the IRM courses for the distance learning format, including production of the video-taped lectures, senior faculty release time and salaries, development of graphics and workbooks and the essential web sites, and course related software. To enable Heinz to amortize these costs over three years, the partners agreed to direct half of the tuition revenues to reimburse these expenses, and to share the remaining half of the revenues equally. Program operating costs such as marketing, staff and facilities would be shared equally and deducted from the second half of the tuition revenues. As an additional incentive for SIPA, the partners agreed that up to five SIPA students would be admitted on a competitive basis to all IRM courses. These students paid Columbia tuition and received Columbia credit for the IRM courses. The partners were primarily motivated by mission, not any expectation of any significant margin of revenues over direct expenditures, particularly during the term of the first three year agreement. Throughout the Fall of 1995 and the early Winter of 1996, the schools moved forward simultaneously on the required administrative and legal approvals and the program design and marketing plan. On February 16, 1996, Deans Kamlet and Ruggie signed a formal Letter of Agreement, and approved a Joint Marketing Program. The initial agreement was for three years, through the 1998-1999 academic year. The agreement formalized the administrative and financial points previously described and detailed the file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (6 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education division of responsibilities between the two schools. It capped each course at 25 students, including the five full-time SIPA students, based on the findings of the research referred to above on optimum class sizes for the distance learning format and a recognition that this was uncharted water for both schools. At the same time, approximately 20 tuition-paying students for all six courses would be sufficient for the schools to cover most of the costs of the experiment. While the primary responsibility for the academic aspects of the program logically fell to Carnegie Mellon, the physical location of the program and its market in New York City dictated that responsibility for marketing and recruitment be handled primarily by SIPA. As with every other aspect of the IRM program since its inception, the success of the marketing effort has been very much a result of shared responsibility and joint decision-making. Key components of the initial 1996-97 joint marketing program were: ● distribution of brochures, applications and information to a targeted mailing list of alumni of both schools, top companies and governments in the New York Metropolitan region and readers of technology journals and in response to requests via the telephone, mail, e-mail and in-person drop-ins; ● development and placement of advertisements in The New York Times and The Wall Street Journal as well as other low or no cost media venues; ● two evening open houses at SIPA, hosted by Eimicke and Pearson (with CMU faculty and program staff, as available) including a brief formal presentation, showing of the promotional video, a free-wheeling question and answer session, facility tours, refreshments, and informal, one-on-one conversations; ● a staffed telephone information line and a dedicated, 24 hour voice mail information/application request telephone line; and, ● individual facility tours and on-site interviews with SIPA coordinator Eimicke upon request. The information line was up and operating by mid-March 1996. An important ingredient in the program's initial success was the hiring of a talented MPA graduate student, Neil Amos, whose energy, dedication, and helpful nature set a customer-friendly tone that sustained the effort through the difficulties of launching the program. The marketing plan was implemented on schedule and about 70 prospective students attended the April 23rd and 24th, 1996 Open Houses. Amos, Eimicke and Pearson were available throughout the summer of 1996 to respond to mail, e-mail and telephone requests, follow-up with all open house attendees, contact major companies and governments throughout the region on a "cold call" basis, and call again anyone who called or contacted us more than once about the program. The program received a financial and psychological boost in the Spring of 1996 when former SIPA Dean Harvey Picker responded favorably to Dean Ruggie's suggestion that Picker make a gift to the school to cover some of the IRM program's start-up costs. The gift helped draw attention to the effort and relieved some of the pressure program staff were putting on themselves to get up to full enrollment file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (7 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education immediately. In marketing the program, it was immediately noticed (and subsequently borne out in discussions with students) that those interested in the IRM program differed in one major respect from the type of student that was typically attracted to SIPA's and the Heinz School's full-time programs. Students generally gave a much later expression of interest in and commitment to the program than is the case for most graduate programs. It was only during the summer of 1996 and 1997 that most students completed their applications and committed themselves to the program. This is because they were on average older, with more family and work commitments than most graduate students and were unable to commit to the program very far in advance. Also, the design of the program purposely sought to minimize work and family disruptions, and so students did not have to make extensive plans to move or leave employment. The first semester of the CMU/CU Joint Program in Information Resources Management was successfully launched on September 7, 1996, with 15 students present at the initial session in New York City. Only one SIPA student applied for the first two courses but she subsequently dropped them as too burdensome in light of her family responsibilities and part-time status. Several other students dropped out, some started late and others were forced to interrupt their studies over the course of the first three semesters for a variety of reasons, including company buy-outs, new jobs, moves, illness, death of a family member and financial problems. The first class had a good mix of young and not-so-young participants, technical and management types, from the city and its suburbs. They came from the financial sector, health care, education, the utilities industry and the computer business. Only a few students had problems with the course content and extra effort by the Heinz faculty and the program's administrative staff resulted in no drop-outs or drops for failure or inability to do the work. Eleven students completed the first two IRM courses, all of them new to the two schools and all of them part-time. Nine part-time students and five SIPA full-time students took the two Spring 1997 courses. Eight students took the two 1997 Summer courses and received their certificates at a festive graduation ceremony (including family members) held in a Columbia University neighborhood restaurant during August 1997. Several other students from the first class are expected to complete all six courses and receive their certificates in 1998. Recruitment for the 1997-98 program began in the Spring of 1997 and it followed the 1996 Marketing Plan for the most part, utilizing advertisements in The New York Times and The Wall Street Journal, two open houses at SIPA, targeted mailings to alumni, employers and all those requesting information in 1996 or 1997. A very successful 1997 innovation was an informational, large color postcard sent to the mailing list of Information Week, one of the field's most widely read magazines. Over 400 requests for information were received by mid-summer of 1997, about double the number of requests in 1996. We believe the increase was primarily due to the postcard mailing and word of mouth recommendations. The partners are cautiously optimistic that the availability and potential value of the IRM program is file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (8 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education becoming more widely known. Over the summer of 1997, we also called every person who had requested an application over the past two years for whom we could locate a telephone number, to encourage them to apply in 1997. A survey of this population revealed that the reasons for not applying were: tuition is too high (37%); background is not adequate to handle the work (self assessment, 22%); and desire for more professionally targeted courses (such as medical applications, 20%). So far, the net results are encouraging, as there were 21 students at the first session for the Fall 1997 IRM courses. Evaluations of the Program The IRM program is not unique. All of its strategies and tools have been used before and others have experimented with similar certificate initiatives. What may be notable about the IRM program is the combination of innovations. The IRM program is illustrative of the commitment of Columbia and Carnegie Mellon Universities to use information technologies to provide quality graduate education programs to working professionals who cannot easily attend conventional degree offerings. Among the more interesting, non-traditional aspects of the IRM program are: Flexibility of Learning- For the most part, IRM students can learn when and where it is most convenient for them. Each course begins and ends with face-to-face instruction at Columbia University but the sessions are scheduled on Saturday to avoid conflicts with the students' full-time jobs. More significant, the bulk of the instruction is conducted through the viewing of video-taped class lectures and completion of assignments/communication via the World Wide Web. Real World, State of the Art- The program seeks to teach the most up-to-date management and analytical aspects of the leading technologies being applied in the workplace, including PC Oracle, ArcView, System Architect, and Expert Choice. Professionally produced video-taped live class lectures, edited for quality, provide most of the visual instruction. Students may also meet in a class room at SIPA to work on projects and assignments but they can also access each other, faculty and program administrators on a 24 hour basis by e-mail, their own chatroom, video bulletin board, and dedicated voice mail telephone lines from anywhere in the world. Bridge To a Graduate Degree- The six course program is accepted for credit at both Carnegie Mellon and Columbia University. Offered over a calendar year, working professionals can complete their concentration requirements for either CMU's MPM or CU's MPA degree. While the degree candidate would subsequently need to be in residence for a year at either one of the two universities to complete the master's degree, they would be about halfway through upon completion of the IRM certificate. For many older students, two year, full-time master's programs are not feasible. The IRM program is making a master's degree more accessible to these students. A Full Partnership- The IRM program represents a true partnership between the Heinz School and SIPA, both ranked among the best of their field. The IRM courses are available to part-time and full- time students at both universities for credit at either university. Part-time students can use the courses as a springboard to a master's degree at either school as a fulfillment of their concentration requirement. file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (9 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education The face-to-face sessions at Columbia University are staffed and managed by faculty and administrators from both institutions. Students can use the computer lab at Columbia University to do their course work "at" Carnegie Mellon. Some IRM students based in New York City have even completed joint projects with full-time students at Carnegie Mellon in Pittsburgh. Personal Contact- In addition to the flexibility and innovations available through the distance learning format, the two schools are doing everything possible to provide IRM students with maximum opportunities for personal contact and traditional course interactions. The Heinz school faculty and teaching assistants are readily accessible and have experimented with several different mechanisms, such as an organized conference call and regular "on call" times, to facilitate communication. SIPA's IRM coordinator and deputy coordinator are available to students on-site in New York City at least five and usually six days a week in person and 24 hours, year-round, by voice mail and e-mail for questions, trouble-shooting, problem-solving, and emergencies. Students are free to use all SIPA class rooms and facilities to review materials, consult with facilitators, work with fellow students and do course work. Fast-Track Admissions- The IRM's small size and specialized nature enabled the partners to offer a separate admissions process which is flexible, fast and user-friendly. Students can obtain information and applications from either school by e-mail, the World Wide Web, fax, regular mail or voice mail, 24 hours a day. Information is mailed within 72 hours of the initial request. The application itself is thorough but simple and concise. Students must submit their undergraduate transcript and three letters of recommendation. A personal interview with the SIPA coordinator is also required but can be done by telephone where distance is a significant obstacle. Applications are accepted throughout the year. Candidates are encouraged to submit their applications as early as possible. While prospective students may apply for less than all six courses and may therefore start with any of the three semesters, preference is given to certificate candidates and they are strongly encouraged to take the courses in order of their sequence, beginning with the Fall semester. To date, all admission decisions have been made in less than two weeks of receipt of the completed application package. One course of transfer credit is permitted. Experiences and Perspectives This section reflects on the different experiences and perspectives of the various groups of people involved in the first year of the program: the faculty, students and administrators. The Faculty The Heinz School faculty reacted with enthusiasm to the prospect of being involved in an innovative teaching experience, although this was often mixed with some trepidation about entering uncharted territory. The program calls for a very intensive front-loaded time commitment in preparing and recording classes and developing teaching materials. Gone are the days when an assignment can be pulled off the photocopier five minutes before class begins. While this has led to very well-produced file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (10 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education classes and materials, it has also resulted in some inflexibility and difficulties in adapting material to reflect students' interests. All the materials used for conventional classes have had to be substantially changed to make them suitable for the World Wide Web and to exploit the interactive environment that the Web offers. Instructors used more graphics and other visual exhibits during the video taping than they had in their prior conventional offerings of these courses. Assignments have had to become much more specific because of the limited opportunity distance learning offers to explain what is required. All the faculty consulted for this paper commented on how the IRM program has forced them to consider at every step how their teaching and their materials would be used outside of the classroom. For some, seeing their performance recorded for posterity on video led to changes in lecture style. Some faculty were concerned that teaching the IRM program would not be as positive an experience as conventional teaching because of the fewer opportunities it provided to be in contact with the students and get to know their interests and concerns. As it turned out, most faculty came to know their IRM students very well, as they were often called and e-mailed. Some commented that the distance learning format perhaps made students less inhibited about contacting faculty outside of the "classroom" and encouraged discussion. Many considered that they had more interaction on a daily basis with their distance than their on-campus students. However, it was certainly the case that the distance learning format required faculty to make a special effort to ensure that students were kept in the loop and up-to-date, often by creating an environment for communication between each other. Steps taken included distributing home phone numbers and being "on call" to answer questions and deal with any concerns. Most phone calls lasted only a few minutes, but sometimes faculty and students would make arrangements to call at a specific time when both were sitting in front of their computer and could work through a problem together. The Students The students evaluated every aspect of the classes and program at the end of each semester and commented in very positive terms on the professionalism of the materials, the program's content and the accessibility of the faculty and teaching assistants. In particular, they valued the flexibility provided by the program's structure. One student, a consultant, traveled frequently and used to take the videos on the road with her. She would wile away the evenings in hotels throughout the United States watching the classes and submitting her assignments from her laptop computer. Another student was called away on urgent business to Europe, but managed to continue the program while he was there. Most commented that they would have been unable to take the program if it had required attendance at traditional classes. To reap the benefits of distance learning, the students commented on the need to be highly self- motivated. The program demanded independent work with minimal supervision, and many said that they found it much easier to fall behind in the work than with conventional classes, despite regular deadlines for assignments. Those who were able to keep up usually did so by making their own schedule and file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (11 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education allocating specific times each week (usually at night, when other family members had gone to bed) to watch the videos and work on assignments. Some students were frustrated by technical problems that they encountered, particularly at the start of the program when they needed to load software and be hooked-up remotely with the Heinz School and SIPA's computer facilities. Usually, the students with the most problems were those with the least technical experience, but all students at one time or another experienced technical glitches, particularly in submitting assignments. SIPA students also completed evaluations of the classes that they took. Their comments are useful in that they provide some insight into how the program's classes compared with traditional classes. Most thought that the class matched the other classes that they were taking at SIPA in terms of content and work-demands. They liked the organization of the classes and the marketability of the skills and knowledge taught. Most said that they would definitely recommend the class to other students. At the same time, however, they recognized some of the challenges of distance learning. In particular, they felt that they were working in isolation and had to make more of an effort to benefit from discussing issues and problems with their peers. Nevertheless, they often commented on the accessibility of faculty and teaching assistants. The Administration The program's format and the partnership between the two schools presented new challenges for those responsible for administering the program. With staff in Pittsburgh and New York, phone calls and e- mails, rather than face-to-face meetings, were the primary means of communication. Because of this, administrative staff found a need for greater emphasis than usual on clarifying responsibilities and timelines for the production and delivery of materials. This did not prove to be easy, given the long lead- in times needed for production and the late receipt of applications. Key Challenges: Managing Communication and Achieving Returns A number of issues and challenges arose during the first year of the program which led us to make adjustments during the first and second years of the program. In many instances, the issues themselves arose as a consequence of the principles and objectives we sought to implement and the technologies we selected to deliver the program. This section covers in more detail the key issues of managing communication and interaction that arose during the first year of the program and achieving a return on our investment. Managing Communication We knew from studies of other distance education programs that preparation and structure were keys to the success of the program. We also knew that support services for the distance learner were important, and both institutions committed themselves to provide these. Despite this understanding, we stumbled at file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (12 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education the beginning in designing an orientation much as we might have for a conventional program. Administrators spoke as might lawyers at a the closing of a real estate transaction about the unique and exciting features of the house. Faculty described the architecture. We spent too little time with tradesmen, demonstrating to the students how to use the power tools the program would need to realize the lawyers' and architects' vision. We realized after the first semester began that we had wasted an opportunity in the program's initial face- to-face session to provide hands-on work concerning such mundane issues as how to submit assignments electronically or how to install the System Architect software that students would be using in the Analysis and Design course. We also quickly discovered that, although we had selected the World Wide Web as the principle vehicle for communication because of its support of alternative platforms-- Windows, Macintosh, and Unix-- we failed to appreciate the fact that supporting communication and application software varied according to students' operating systems. The fix to some of these problems is relatively straightforward: require students to use a standard operating system (Windows 95), and to change the Orientation sessions into a hands-on workshops in the computer lab. These changes were made for the second year of the program. However, issues concerning the communication technology extended beyond these teething problems. Although the program required experience in information systems and a familiarity with database structures, we underestimated two aspects of the learning environment which are available to conventional students but not the distance learner: (1) opportunities to learn "over one's shoulder"; and (2) an infrastructure of campus computer labs where applications software are readily available and maintained for students by offices of computer and information services. An interesting paradox arose during the first year of the program. On the one hand, students reported that they interacted far more with faculty and staff in this program than they had in any other prior undergraduate or graduate program. On the other hand, despite the ease with which the technology enabled interaction, they felt isolated. This isolation, both perceived and real, had an important consequence for the way in which students learned and the demands they placed on the program. Students frequently face irritating roadblocks in their mastery of information systems materials because of problems in software applications. Programs do not always behave as you would expect them to, and troubleshooting can be time consuming. In the conventional computer lab, students facing such a problem often turn to the student next to them or "over the shoulder" and ask if the neighbor has been experiencing similar problems. The neighbor looks at the distressed student's screen, pauses, and then says something like: "Oh, you forgot to hit the Enter key." The key is struck, and the student sails on, stress relieved. Distance education students do not have these opportunities for informal learning and quick and simple fixes to minor technical problems. They can e-mail or call a teaching assistant for help. Over the course of the first semester, we learned how important this service was for our students. But our teaching assistants are imperfect substitutes for the informal and incidental learning that takes place between file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (13 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education students in conventional classroom formats. A corollary of this observation is that our distance learning students began to expect very short response times. We heard from them when our staff did not respond within 24 hours. We also discovered that our distance learning students contacted faculty or teaching assistants instead of each other for help much more than we had expected. We also took for granted the value of having a computational infrastructure in place and on-site at our respective institutions. Full-time residential students walk into a computer lab, sit at a terminal, enter their user identification and password, point and click on a software application, begin their assignment. Our distance learners do not and cannot do so, unless they commute into the labs at Columbia University, thereby somewhat diminishing the value of the distance education format. Each distance learner in our program became his or her own Office of Computing Services. While there is some value in learning how to install and maintain software, that was not a learning objective of the program. Instead of mastering the course content, distance learners were spending time debugging installation problems. Achieving Returns on Investment There is very little information available in the literature and at conferences on distance education on the returns to investment in such programs. We will not rectify this situation in this paper. Our experience is too limited to provide any conclusive evidence. However, we can review the expenses and revenues associated with our program as background for those who are considering the development of distance education programs. The expenses associated with the program can be divided into two basic components: (1) development and (2) yearly operating costs. Revenues are derived at the moment only from tuition. The Heinz School provided resources for summer salary and videotaping support to a senior faculty member to develop and implement a pilot curriculum development project two summers prior to the targeted launch of the full program. We followed two principles here: (1) the program itself would require senior faculty leadership; and, (2) participating faculty members would learn how to develop these materials through a discovery-based process for which we provided resources. This pilot project cost approximately $20,000, all supported by funds provided by the Heinz School, as was the development cost of the entire program. The direct cost for developing the materials for the six courses were largely those associated with creating the videotaped lectures. The Heinz School used CMU's video recording studio at the Software Engineering Institute (SEI) for this purpose by recording conventional classes as they were being conducted. The facility includes a classroom that can seat from 20 to 50 students. It has four cameras and desk microphones that are remotely controlled by two operators. The facility itself cost approximately $1 million to build and equip more than five years ago. The use of this facility resulted in the production of professional quality, edited videotaped lectures that cost approximately $500 per file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (14 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education finished hour. We collected other estimates using different production facilities which we found charged from $1600 to $30,000 per finished hour of videotape. Overall, each course required approximately $25,000 to videotape, or $150,000 for the entire curriculum. The principle annual operating expenses associated with the program include: salaries-- $200,000; travel-- $30,000; reproduction of materials-- $30,000; and marketing--$50,000. The annual costs of operating the program total approximately $310,000. These annual operating expenses are offset if the program enrolls 22 students at the $2,400 per course tuition that is charged part-time students in one of the Heinz School's conventional degree programs. The initial development costs are recovered by enrolling another 14 students. The first year enrolled 10 full-time equivalent students and the second year an additional 14. We believe that all universities will eventually adopt some form of distance education program. Information technologies themselves will push institutions into such programs. The need for older learners to continue or renew their learning in settings which differ from conventional classrooms will pull them. The questions, if we are correct in this regard, are not whether, but when and how institutions will develop and administer distance education programs. Surely, the cost of developing today's distance education program will continue to fall, and the availability of electronic communication technology will continue to reach broader segments of the population, thus expanding the potential audience for these programs. The cutting edge will remain expensive. Implications for Extensions There are real possibilities for further collaboration in distance learning along the lines pursued in the collaboration described in this paper. Many programs are unable to deliver a given area of instruction due to resource constraints. In addition, as in this case, the area being considered may be important enough, and the need for instruction great enough, that an external mid-career market might be available to cross-subsidize the costs of a distance learning project. The opportunity comes because there is a mismatch between the geographic location of faculty and the market need for their expertise. While Columbia University has some faculty in information management, it does not come close to the depth of expertise in this field possessed by Carnegie Mellon. On the other hand, the mid-career market for this expertise is greater in New York City than in Pittsburgh. Columbia's need for faculty in this area and Carnegie-Mellon's desire to ease their access to a larger mid-career market created the conditions that led to this collaboration. One of the main contributions that Columbia made in the collaboration with Carnegie-Mellon is to provide local marketing and facilities. Even if the program was delivered without in-person instruction, use of a local host is probably desirable. The potential for similar collaborations between universities is quite real. While the start-up costs of the Carnegie-Mellon-Columbia collaboration were high, video taped lectures could be produced without broadcast quality production values, and costs could be lowered if necessary. Universities might develop file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (15 of 17) [11/4/2008 6:54:24 PM] A Case Study In Distance Education two-way exchanges of faculty and virtual visiting professor programs that could increase learning opportunities for students at a fraction of the cost of hiring permanent in resident faculty. Another possibility is that organizations such as NASPAA or APPAM could earn income and provide a service by brokering distance learning course consortia. A school could offer a video-taped/Internet based course that could be offered by perhaps as many as five other schools simultaneously. Each university could "sell" the course to its own students or local mid-career market, and offer its own academic credits for the course. The number of students in the course would be capped at a set number, to give faculty and TAs sufficient time to grade work and interact with the class over the Internet. The institution providing the instruction would receive a fee or royalty for each student enrolled. Offering course credit for distant learning courses would require approval of the course and faculty from each university. That approval process was necessary in the Carnegie-Mellon-Columbia Information Resource Management Program in order for Columbia University students to apply the credit for the program's information management courses to their degree. For a multi-university consortium arrangement to be practical we would need to go beyond the mixed distance/local learning experiment we have conducted to a complete distance learning program. The Internet, FedEx, and credit cards make this technically feasible. While we believe the mixed system we have piloted in New York is quite successful educationally, the potential for additional collaboration might require a further leap to a pure distance learning experiment. A course offered without personal appearances by distant faculty could be team taught by local faculty, or could be a 2-to 4-week module imbedded within a longer course. For example, a course in public management might have a two week module in information management provided by Carnegie-Mellon. Another way to reduce travel costs might be to have the opening session of a distance learning class introduced by local faculty or administration who could distribute video tapes, assignments, or software. Costs could be further reduced by providing all of these materials through overnight package services and the Internet. Creative use of this technology could make it possible to open up learning experiences to students from faculty situated in any part of the globe. Courses in international issues might be taught by faculty from outside the U.S. or might be taught in other countries by faculty from U.S. universities. The potential for distance learning is only limited by our imagination. While the technology requires new teaching methods to insure quality, if care is taken, distance learning courses can provide unique, high quality educational experiences. Back to Personal Homepage SIPA Directory file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (16 of 17) [11/4/2008 6:54:24 PM] file:///C|/Users/admin/Documents/websites/cohen/notused/cohen.html http://www.columbia.edu/cu/sipa http://www.columbia.edu/cu/sipa A Case Study In Distance Education Columbia University Home Page SIPA Web Manager file:///C|/Users/admin/Documents/websites/cohen/notused/cmrevisedjpae.html (17 of 17) [11/4/2008 6:54:24 PM] http://www.columbia.edu/ mailto:hm3@columbia.edu Local Disk A Case Study In Distance Education