Introduction The American Journal of Distance Education, Vol. 13 No. 1, 1999, pp. 25-43 Methodology for Cost-Benefit Analysis of Web-Based Tele-Learning: Case Study of the Bell Online Institute Tammy Whalen and David Wright Abstract Educators, trainers, and business people need to be able to evaluate the cost-effectiveness of Web-based training in order to make informed decisions about the extent to which this new media should be used in their organizations. The present study hypothesizes that there are several key design elements that should be considered in costing Web- based training projects. The relative importance of these elements is examined using a case study approach. The methodology used in this case study can be employed in future cost benefit studies of Web-based training. This case study also provides a detailed cost-benefit analysis, including the breakeven number of students required to recover Web-based course development costs and the return on investment over a five-year period. Introduction Web-based training is receiving a great deal of interest in academia and private industry, and cost analysis has become increasingly important. Many universities such as the University of Phoenix, the University of Maryland, and Athabasca University, to name a few, have multimillion dollar budgets for Web-based teaching of part-time students. In private industry, Bell Canada is a good example of how companies are using Web-based learning for both in-house employee training and as a service product offered to customers who want to apply distance learning in their own organizations. In both cases, the cost benefit of Web-based training must be established. Internally, Bell Canada is interested in using Web-based training because it is a cost-effective and convenient alternative to classroom training. Since Web-based training is also a customer product, it is necessary to demonstrate its value in order to interest customers in using this type of training model. For these reasons Bell Canada undertook a pilot project to examine the cost-effectiveness of Web-based training. The literature in the area of cost-benefit analysis of Web-based distance learning is sparse, and the objective of this paper is to fill the gap in the literature. The costing methodology discussed in this paper was developed by the authors and tested in the Bell Canada case study. The results of the cost-benefit analysis provide further information about the economic benefits of Web-based training. Measurements of Financial Performance Although few cost-benefit analyses for Web-based training are available, standard measures of financial performance apply to this type of study. Two common measures are the breakeven point, the point at which costs are recovered, and return on investment, which illustrates the economic gain or loss from having undertaken a project. Breakeven Number of Students. To offset the high fixed costs of Web based courses, a certain number of students must be trained at a delivery cost per student of less than that of the delivery cost per student for classroom training. The number of students that offsets the fixed costs of Web-based training is the breakeven point. In Figure I the total fixed costs for classroom and Web-based courses are represented by the intercepts of the vertical axis. The cost of course delivery per student is represented by the slope of each line. Since Web-based courses cost more to develop, the line meets the vertical axis at a higher point than for classroom costs. However, since Web-based courses cost less to deliver per student, the slope of the line is more gradual. The point at which the lines cross is the breakeven number of students needed to recover the higher fixed costs of Web-based delivery. The American Journal of Distance Education, Vol. 13 No. 1, 1999, pp. 25-43 Return on Investment. The return on investment (ROI) is the percentage that represents the net gain or loss of using Web-based training instead of classroom delivery. For example, an ROI of 300% means that $3 was saved in reduced delivery costs for every $1 spent on Web-based training. Cost Comparison: A Background In comparing Web-based distance education and classroom course delivery, there must be a clear basis for measuring output. It can be argued that the true measure of training output is the information and skills acquired by students. However, many studies in the literature report that there is no significant difference in the amount of learning that students acquire through classroom training and distance learning (Russell 1998). For the present costing analysis, the basis for comparison will be the costs of development and delivery of the courses themselves. In analyzing the costs of Web-based courses, the basis for comparison is the cost of delivering an equivalent course in the classroom. It was assumed that the learning outcomes for the Web-based courses in this study would have been the same if the courses had been delivered in a classroom. Costs are divided into fixed and variable costs in order to compare technology-enabled learning to traditional classroom delivery. Fixed costs are defined as costs that remain the same regardless of the output. Variable costs are those that vary directly with the amount of output. Thus, variable costs increase with the number of students, while fixed costs are incurred before a course is even offered. Clearly, costs that would be incurred even if a course were delivered in a classroom (i.e., sunk costs) should be ignored in costing Web-based training. For example, if a course were previously offered in the classroom, and no further research and development for course materials were needed, the cost of creating the intellectual property for the course would be a sunk cost. Likewise, the cost of feasibility studies should not be included as a costing element. In an analysis of distance education using video conferencing-based delivery at the University of Ottawa, Aub6, Chilibeck, and Wright (1996) established the fixed costs for videoconferencing as: videoconferencing equipment, technicians' salaries for running the equipment, installation costs, and fees for basic telephone lines. Variable costs for videoconferencing delivery of distance courses included fees for long distance network usage, shipping charges for supplementary print materials, honoraria for professors, and salaries paid for the preparation of course materials. The breakeven point, which was found to be twenty two courses with the videoconferencing configuration used at the University of Ottawa, is the financial measure used in the analysis. Trevor-Deutsch and Baker (1997) reported on the costing of a course with multimedia elements delivered via videoconferencing in an academic setting. They considered the following costs for traditional classroom delivery: the instructor's salary and benefits; the number of courses taught by the instructor; and the costs of course The American Journal of Distance Education, Vol. 13 No. 1, 1999, pp. 25-43 development, course materials, administrative support, classroom overhead, and any additional time the instructor spent on the course for activities such as grading and meeting with students. Costs for the same course delivered at a distance included equipment costs and course development costs. The breakeven point was calculated at 331 students. Future courses using existing equipment and some of the development features of the pilot course were estimated to have a breakeven point of eighty-two students, a 75% reduction in the breakeven point of the pilot. Another study compared CD-ROM-based learning to classroom learning in a high-tech industry setting (Hall 1997). Costs for traditional classroom delivery included development time, classroom overhead costs, instructors costs, and travel costs for the participants. The CDROM training costs included equipment and course development time. Development time was high for the CD-ROM course, totaling $1,205,394 over three years. However, the classroom delivery costs were also high, primarily due to high travel costs for the students and the length of time spent away from the job. Costs over the three-year pilot period for the CD-ROM-based course were 47% less than those for the classroom-based course. The payback period was fifteen months, and the Internal Rate of Return-the discount rate that makes the Net Present Value of the investment equal to zero-was 61 %. The breakeven number of students needed to recover development costs was not calculated. The course material was compressed by 60%, from four days in the classroom to 11.2 hours for the CD-ROM version. The reduction in training time was attributed to improved instructional design, the ability of students to test their knowledge and then omit some sections of the course, and the variety of instructional models such as text and animation available to students, which contributed to more effective learning. Several other publications are relevant to the discussion of Web-based costing analysis. In a recent article, Hall (I 998) itemized the current contractor rates for the types of experts required for Web-based course production. Bates (I 995) also discusses the costing of distance education courses, although the Web is not included in his analysis. Several bibliographies on both return on investment and evaluation have been compiled by the American Society for Training and Development (I 998) and are available on their Web site at . Another report by the Higher Education Information Resources Alliance (1994) primarily examines non-economic benefits of the use of information technology in an academic setting. While all of these studies are useful in providing a comparative look at methods of determining the costs and benefits of distance education, few reports deal specifically with the economic analysis of Web-based courses. This study takes a new approach in looking at cost-benefit analysis for the new delivery method of the Web. Costing Methodology Web-based training has become a widely explored topic in the education and training literature, but there are still few comprehensive, tested, costing methodologies available for use by educators, trainers, and business people who need to make decisions about the cost-effectiveness of Web-based training for their own organizations. This study hypothesizes that there are several key design elements that must be costed in a majority of Web-based training projects. These costs are divided into fixed capital costs and variable operating costs. Capital costs include the server platform shared by all courses mounted on that server as well as the cost of the content development shared by all students taking that course. Content development includes six items: 1) instructional and multimedia design; 2) the production of text, audio, video, graphics, and photographs; 3) the development of authoring and delivery software, or the cost of licensing commercial software;4) the integration, modification, and testing of course content; 5) student and instructor training; and 6) course testing. Operating costs represent the costs for the time students and instructors spend using the courses. These costs are analyzed to determine the costs per course, the costs per phase of development, the costs per student, and the costs per mode of delivery (i.e., synchronous or asynchronous). The Ratio Analysis section of this article provides an evaluation of the breakeven number of students required to recover course development costs and the return on investment over five years. The costs of producing future courses using the same instructional design or the same mode of delivery can be estimated from the results of this study. To verify the costing methodology, it is necessary to determine whether some of these costing elements are more important than others. In other words, do some elements dominate others, or are some elements insignificant? To test the hypothesis that all these elements are critical to costing analysis for Web-based course design and delivery, the costing methodology is applied to the following case study. Bell Online Institute Case Study The Bell Online Institute (BOLI) is the Bell Canada business unit that delivers Web-based training to Bell Canada employees. BOLI also functions as a testing ground for various Internet-based learning platforms used for delivery The American Journal of Distance Education, Vol. 13 No. 1, 1999, pp. 25-43 of training at Bell Canada. A separate business unit, the Bell Institute for Professional Development (BIPD), provides traditional classroom course delivery and oversees all internal training. The Bell Online Institute case study was undertaken to measure the cost and evaluate the effectiveness of training delivered on four different Web-based learning platforms: • WebCT (http:Hwww.webct.com/) developed at the University of British Columbia for asynchronous training; • Mentys (http:Hwww.globalknowledge.com/) from Global Knowledge Network for asynchronous training; • Pebblesoft (http:Hwww.pebblesoft.coml) for asynchronous training; and • Centra Symposium (http://www.centra.com/) for synchronous training. Three courses designed for delivery on these learning platforms were produced by three independent vendors. All of the courses were on telecommunications topics: TCP/IP (using the WebCT and Mentys platforms), frame relay (using the Pebblesoft platform), and routing (using the Centra Symposium platform). The courses were estimated to be equivalent to two-day classroom courses. The pilot courses were delivered to engineers working in Bell Canada's Advanced Communication Systems group. Three of the learning platforms, WebCT, Mentys, and Pebblesoft, present course materials asynchronously. That is, course materials reside on a server on an on-going basis and may be accessed at the student's convenience. The frame relay course on the Pebblesoft learning platform was authored in French. The fourth platform, Symposium, is a synchronous learning platform using 28.8 Kbps delivery over the Internet or an intranet. The system supports text, graphics, and animation to present course materials. Shared features for system users of Symposium include audio communication among the course participants and the instructor, an electronic whiteboard, an Internet browser, and a live text chatroom. Cost Analysis: Fixed Costs This portion of the analysis helps determine whether the high fixed costs associated with providing learning in a technology-enabled format are justified in comparison to the costs of traditional classroom delivery already provided through BIPD. The actual billed rates reported by the vendors are used in the cost charts found throughout the paper. All costs are reported in Canadian dollars. Capital Costs. License Fees for Learning Platform Software. The price of a learning platform depends on the number of people who will be using the software and ranges widely from vendor to vendor. An upgrade cost of 10% per year is assumed. Using 10% of the costs as an annuity, the present value of the upgrades and the original purchase price are shown in column three of Table 1. The cost of the platform must be amortized over the total number of courses that will use the platform. BIPD presently has 150 independent study courses on CD-ROM or diskettes. It is assumed that all of these courses could potentially be taught using any of the asynchronous platforms. There are presently 700 classroom-based courses offered through BIPD. It is estimated that 10% of these could potentially be offered using the Centra Symposium synchronous platform. Table I summarizes the costs associated with learning platforms. The American Journal of Distance Education, Vol. 13 No. 1, 1999, pp. 25-43 Hardware. In this study, it is assumed that a server must be purchased to offer the course, but that client computers are already available on the desktops of the employees participating in the course. The purchase price of the server was estimated at $15,000. The total number of courses, as established previously, is 220 (150 asynchronous + 70 synchronous). It is assumed that not all courses would need to run on the server at one time. By amortizing the cost of the, server over the number of courses, the cost of the server per course equals $68 ($15,000 / 220). On average, it is estimated that each asynchronous course requires 100 Megabytes of memory. A synchronous course would typically require less interactivity with the course material because of the presence of a live instructor. Therefore, the course would include less multimedia and require only twenty-five Megabytes of memory per course. During the pilot study, staff time was required to set up desktop machines-for example, the installation of browser plug-ins-to take advantage of the multimedia aspects of the courses. However, this cost was not included in the study since newer versions of the learning platforms will automatically install the software needed for the desktop computers. Total Capital Costs per Course. The capital cost per course is equal to a server cost of $68 ($15,000 / 220) plus the platform costs per course for each different platform: - Synchronous routing course: $752 ($68 + $684 for Symposium) - Asynchronous TCP/IP course using WebCT: $95 ($68 + $27) - Asynchronous TCP/IP course using Mentys: $1,436 ($68 + $1,368) - Asynchronous frame relay course using Pebblesoft: $1,664 ($68 + $1,596) Phase 1: Content Development. Content development variables include: instructional design; multimedia design; the production of text, audio, motion video, graphics, and photos in machine-readable format; course authoring; software development; integration of content and testing; modification/adjustment; training; and course testing. Only one of the courses, TCP/IP, included motion video elements. Developer Salaries. The billed hourly rates for the independent contractors were used in the analyses found throughout this paper. These are the rates that customers of these companies were willing to pay for Web-based course development. Differences in Synchronous and Asynchronous Course Development. Significant differences were found between the asynchronous courses and the synchronous course regarding course development costs. The asynchronous courses were similar in the types of tasks performed by developers and the total amount of time spent on course development. The synchronous course, however, required far less development time, primarily due to less use of multimedia. For example, the synchronous course contained no audio, video, or photographic elements and also had fewer graphics. An average of 1,321 development hours were spent on the asynchronous courses, while only 144 development hours (89% fewer) were required for the synchronous course. Since development costs depend on the number of hours required, the synchronous course cost much less to develop. Cost Analysis: Variable Costs Phase 2: Usability Testing. Phase 2 of the BOLI Pilot Project involved course testing in which Bell Canada employees participated in training using the Web-based courses. This represented the total variable costs for the project. Since Bell Canada must pay employees for the time they spend in training, student salary costs are a significant factor in this costing analysis. More time spent in course delivery translates into higher student salary costs and less cost savings. The salary costs of students and, for the synchronous course, of both students and the instructor, were analyzed to compare the delivery costs per student. Figure 2 gives an overview of the delivery costs for all courses. These costs are described in more detail in the following section. The American Journal of Distance Education, Vol. 13 No. 1, 1999, pp. 25-43 Table 2 gives a summary of the fixed and variable costs, for each Webbased course in the study as well as the baseline classroom course costs. A detailed analysis discussing how these numbers were derived is found in the following section. Table 2. Fixed and Variable Costs for Web-Based Courses Course Total Fixed Costs Total Variable Costs Classroom $28,600 $1,284 ($600 + $614 + $70) Routing $24,332 ($23,580 + $752) $242($176 +$66) Frame Relay $88,364 ($86,700 + $1,664) $110 TCP/IP on WebCT $158,708 ($158,613 + $95) $110 TCP/IP on Mentys $160,049 ($158,613 + $1,436) $110 Cost Analysis of Courses Classroom Courses. Life-Span and Duration of Courses. The life-span of classroom courses is five years, meaning that no upgrade of course material is expected within a five-year period. The duration of classroom courses is fourteen hours. Although the actual time spent in the classroom is approximately twelve hours, the course participants are paid for two normal seven and a half hour work days. Costs of Classroom Training. Although none of the Web-based courses in this pilot project had been offered previously in the classroom, it was estimated that the courses were equivalent to two-day classroom courses. Therefore, the costs of delivering these courses in a classroom setting can be calculated by using the figures for delivering a typical two-day classroom course at the Bell Institute for Professional Development (BIPD). Course Development Costs. For each day of classroom training BIPD requires twenty days of development (20: 1). BIPD charges two different development rates depending on whether or not BIPD will deliver the courses and have the opportunity to recover some development costs from future delivery. If BIPD delivers the course, there is a charge of $715 per day for course development to the business unit requesting the course. If BIPD does not deliver the course, there is a charge of $995 per day for course development. Therefore, the development of a two-day classroom course similar to the ones in the pilot project would cost the customer a total of $28,600 ($715 * 40 days). Tuition Fees. In-house tuition fees are also charged to the participants' department to recover BIPD's normal operating costs. Classroom courses at BIPD have, on average, ten participants. Each participant is charged $250 for The American Journal of Distance Education, Vol. 13 No. 1, 1999, pp. 25-43 tuition and another $50 per person to cover such costs as the instructor, course materials, and refreshments. This totals $300 per day for each participant. Therefore, for a two-day course similar to those in the pilot, a cost of $600 per participant would be charged by BIPD to the business unit in which the participant works. BOLI did not charge tuition to participants in the pilot. However, tuition charges for Web-based courses are anticipated with the full implementation of Web-based training to Bell employees. The costing information in this study will be taken into account to determine an appropriate price to charge business units in the company. Salary Costs for Course Participants. Twentv Bell Canada engineers were enrolled in the synchronous course, each earning an average annual salary of $57,700. The loaded annual salary including vacation, pensions and benefits, a portion of a supervisor's salary, office space, and office supplies and services is $85,500 per year, or $44 per hour. Each of the three pilot courses was equivalent to a two-day classroom course. Each participant was paid for fourteen hours of work for this period, at an average loaded hourly rate of $44. Therefore, the salary cost of offering classroom courses is $614 per participant. Travel Costs. Approximately 10% of employees taking courses at BIPD travel to a different city to take the course. At a typical travel cost of $350 per day, which includes airfare, one night in a hotel, and travel allowances, a two-day course costs Bell Canada $700 per student who travels. If 10% of the students travel, the travel cost per student is $70. Web-Based Courses. Life Span and Duration of Courses. The lifespan of Web-based courses is also five years. The duration of Web-based courses is two and a half hours for asynchronous courses and four hours for synchronous courses. Cou Corn ression. Compression ratios for Web-based courses are 79% for asynchronous courses and 67% for synchronous courses. Hall (1997) points out that there is strong evidence that CD-ROM-based multimedia training requires less time than classroom training. Reported course compression ratios range from 20-80%, with 40-60% being most common for CD-ROM training. In the Bell Online Institute Pilot, course compression ratios were at the high end of the reported ranges in the Hall study for both synchronous and asynchronous courses. The Web-based asynchronous courses lasted two and a half hours, while comparable classroom versions would have lasted twelve instruction hours. Course compression was, therefore, 79% for both asynchronous courses. The synchronous course lasted four hours, while a comparable course offered in a classroom would have lasted twelve instruction hours. This equaled a compression ratio of 67% for the synchronous course. Salarv Costs for Course Participants. Synchronous Course. For one engineer to complete the four-hour synchronous course, salary costs totaled $176 ($44 * 4). There was also an instructor present during the full four hours of the synchronous course. Theoretically, the Centra Symposium system can accommodate fifty participants at one time. However, this large number of concurrent participants would seriously limit the opportunities for interaction among the students and the instructor. For this study, it was assumed that the synchronous routing course would be taught in groups of ten. To calculate the cost of the instructor's salary per student, an hourly rate of $164 was used-the billed rate reported by the course vendor. This equaled a cost of $66 per student ($164 * 4 hours / 10 students). Asynchronous Courses. The asynchronous courses each had 29 participants who were also Bell Canada engineers. Based on the average loaded rate of pay, the salary cost for one engineer to complete a two and a half hour asynchronous course was $110 ($44 * 2.5). There are also opportunity costs involved in having employees spend time away from their work to take courses. For example, the completion of a project may be delayed due to time spent in training, and that delay may have unforeseen consequences for the organization's business. One advantage of asynchronous Web-based training as opposed to either synchronous or classroom training is that these problems are alleviated due to greater flexibility in scheduling students' time. However, although student salary costs are one measure of opportunity costs, total opportunity costs are almost impossible to determine and, therefore, have not been included here. Web-Based Course Development Costs Asynchronous Courses. Frame Relay Course. The frame relay course was one of two courses delivered on an asynchronous platform. The course vendor had difficulty using Pebblesoft because the initial software was delivered without French character support. This caused an estimated 24% increase in development time. If the project had been completed without the software problem, the time spent on course development would have equaled 932 hours, and development costs at $562.50 per day would have totaled $69,900. Table 3 gives the total cost of each developer task at the billed rates reported by the course vendors and also summarizes the number of hours spent on each task, including the percentage of the total that this represents. TCP/IP Course. Another vendor produced an asynchronous course on the topic of TCP/IP. Two different platforms were used to deliver the course: Global Knowledge Network's Mentys system and Web-CT, developed at the University of British Columbia. A staff member of BIPD also participated in course development by designing the The American Journal of Distance Education, Vol. 13 No. 1, 1999, pp. 25-43 content architecture for the instructional design plan. The TCP/IP course included a five-minute video segment and was the only course that included video. Table 4 gives the total cost for each developer task as well as the number of hours spent on each task and the percentage of the total time spent. Table 3. Frame Relay Course (Asynchronous): Cost and Time per Developer Task Developer Task Cost Percentage of Time Training $1,200 1.4% (16 hrs.) Tests $9,975 11.5% (133 hrs.) Instructional Design $18,525 21.4% (247 hrs.) Multimedia Design $13,125 15.1% (175 hrs.) Text Production $6,225 11.5% (133 hrs.) Audio Production $1,875 2.2% (25 hrs.) Graphics Production $12,300 14.2% (164 hrs.) Photo P reduction $1,575 1.8% (21 hrs.) Authoring/Software Development $6,075 7% (81 hrs.) Content Integration $7,800 9% (104 hrs.) Testing Integration $2,550 2.9% (34 hrs.) Modification/Adjustment Integration $5,475 6.3% (73 hrs.) Total cost at $562.50 per day = $86,700. (All costs are reported in Canadian dollars.) Total of 1, 156 hours. Synchronous Course. Routin Course. The routing course was delivered on the Centra Symposium platform. The pilot course was delivered to twenty Bell Canada engineers from Quebec City, Montreal, Ottawa, Toronto, and Hamilton. The course was offered in two parts, one in the morning and one in the afternoon. The morning session lasted two and a half hours and the afternoon session lasted one and a half hours, for a total of four hours. All of the engineers were present for the entire course. Table 4. TCP/IP Course (Asynchronous): Cost and Time per Developer Task Developer Task Cost Percentage of Time Training $7,893 5% (74 hrs.) Video Production $26,667 16.8% (250 hrs.) Instructional Design $36,800 23.2% (345 hrs.) Multimedia Design $9,173 5.8% (86 hrs.) Text Production $747 0. 1% (7 hrs.) Audio Production $4,053 2.6% (38 hrs.) Graphics Production $31,253 19.7% (293 hrs.) Authoring/Software Development $12,907 8.1% (121 hrs.) Content Integration $19,093 12% (179 hrs.) Testing Integration $3,733 2.4% (35 hrs.) Modification/Adjustment Integration $6,293 4% (59 hrs. Total cost at $800 per day = $158,612. (All costs are reported in Canadian dollars.) Total of 1,487 hours. Staff from the Bell Institute for Professional Development assisted the independent course vendor in developing the routing course and co-facilitated the pilot course delivery. Table 5 summarizes the cost, the number of hours, and the percentage of the total time for each developer task. Ratio Analysis Breakeven Number of Students. The breakeven number of students for each course is listed below. • Routing course: 4 students ($28,600 - 24,332) / ($1,284 - 242) The American Journal of Distance Education, Vol. 13 No. 1, 1999, pp. 25-43 • Frame relay course: 51 students ($88,364 - 28,600) / ($1,284 - I 10) • TCP/IP course on WebCT: III students ($158,708 - 28,600) / ($1,284 - 110) • TCP/IP course on Mentys: 112 students ($160,049 - 28,600) / ($1,284 - 110) Table 5. Routing Course (Synchronous): Cost and Time per Developer Task Developer Task Cost Percentage of Time Training $2,620 1 1. 1% (I 6 hrs.) Tests $655 2.8% (4 hrs.) Instructional Design $17,849 75.7% (109 hrs.) Multimedia Production $491 2.1% (3 hrs.) Text Production $1,310 5.6% (8 hrs.) Graphics Production $655 2.8% (4 hrs.) Total cost at $1,228 per day $23,580. (All costs are reported in Canadian dollars.) Total of 144 hours. Return on Investment. Expected Number of Students per year. To calculate return on investment an assumption must be made concerning the number of students that will be trained per year with a course. The number of students trained per year is used to calcul ' ate the fixed costs per student so that these costs can be amortized over all students who will use the course. Otherwise, we would be trying to recover all fixed costs with the first student, which clearly cannot be done and which fails to take into consideration the reusability of both the course itself and the hardware and software used to deliver it. In the present study, it was expected that 30-40% of Bell Canada's 300 Wide Area Network (WAN) engineers would take this course, equaling 125 participants. Another seventy-five participants from other Bell Canada departments are also expected to take each of these courses. Thus, the expected number of students per year was 200 students per course. Platform Costs per Student. • WebCT: $0.14 per student ($27 / 200) • Mentys: $7 per student ($1,368 / 200) • Pebblesoft: $8 per student ($1,596 / 200) • Symposium: $3.50 per student ($684 / 200) Salary Costs per Student for Classroom Training. To calculate the return on investment it is necessary to know the savings per student for Web-based delivery over classroom delivery. Therefore, the delivery cost per student in a classroom must be determined. The cost of offering these three courses in a classroom to 200 participants per course annually would be $122,769 per course or $368,308 for all three courses. Assuming a classroom course would be offered to 200 students per year, the classroom course development costs per student would equal $143 ($28,600 / 200). Table 6 presents an analysis of the savings per student for each course. The return on investment (ROI) for each course in the expected case of 200 students trained per course per year is found in Table 7. Table 6. Savings per Student Classroom Course Costs Course Development $143 Tuition $600 Travel $70 Salary of Students $614 Total $1,427 The American Journal of Distance Education, Vol. 13 No. 1, 1999, pp. 25-43 Web-Based Course Costs Frame TCP/IP TCP/IP Relay Routing Course on Course on Course on Course on WebCT Mentys Pebblesoft Symposium Course Development $660 $660 $434 $53 Server ($15,000/3 courses/200) $25 $25 $25 $25 Learning Platform $0.14 $7 $8 $3.50 Salary of Students $110 $110 $110 $176 Salary of Instructor - - - $66 Total $795 $80 $577 $324 Savings per Student Over Classroom Delivery $632 $625 $850 $1,103 Table 7. Course ROls Course ROI Dollars Saved for Every $1 Spent Routing (synchronous) 3283% $33 Frame Relay (asynchronous) 697% $7 TCP/IP (asynchronous using WebCT) 228% $3 TCP/IP (asynchronous using Mentys) 283% $3 The present value Of the cost savings was estimated to be 3.621, an amount that takes into account the five-year life span of the courses and a discount rate that reflects a moderate investment risk. The ROI was positive in all cases, although the courses with the least amount of multimedia content-the synchronous routing course and the asynchronous frame relay course-had the best ROls. The TCP/IP courses had the highest percentage of multimedia content as well as the highest breakeven points. Table 8 summarizes the number of multimedia development hours per course, the corresponding breakeven number of 'students, and the ROI over five years. The results indicate that all courses will breakeven within the first year if 200 students per year take each course. The American Journal of Distance Education, Vol. 13 No. 1, 1999, pp. 25-43 Table 8. Multimedia Development Hours, Breakeven Points, and Five-Year ROls of Courses Multimedia Breakeven Course Development Number of ROI Over 5 Hours Students Years Routing 144 hours 4 students 3283% Frame Relay 1,156 hours 51 students 697% TCP/IP on WebCT 1,487 hours 111 students 288% TCP/IP on Mentys 1,487 hours 112 students 283% Discussion Comparison of Multimedia Content per Course. There was a significant difference in the amount of multimedia content in the courses and, therefore, in the amount of time spent on multimedia development. The synchronous routing course contained only a few graphics, while the asynchronous courses contained a higher percentage of graphics as well as audio and photographs. The TCP/IP course had the largest percentage of multimedia because it included a five-minute video segment. The percentage of development time spent in multimedia production and design for each course is listed below. * Synchronous routing course: 5% * Asynchronous frame relay course: 33% * Asynchronous TCP/IP courses: 45 % Importance of Each Costing Element. While this is just one case study, there still appears to be evidence that the costing elements chosen for this study are all important. There was some variation in the use of any particular element among the courses, although the variations were small and some elements were consistently used more or less frequently for all of the courses. None of the costing elements could be discounted as unimportant, and none of the costing elements clearly dominated the others in importance. More research needs to be done in the area of costing analysis for Web-based course design and delivery, but practitioners working in the field may still find the conclusions of this study useful for their own cost-benefit projects. Conclusion Web-based training has higher fixed costs than classroom based training; however, these higher course development costs are offset by lower variable costs in course delivery. This is primarily due to the reduction in course delivery time (course compression) and the potential to deliver courses to a larger number of students than is possible in a traditional classroom without incurring significant incremental costs. Realizing savings for Web-based courses requires a sufficient number of students in order to recover course development costs. Since employees must be paid for time they spend taking a course, student salaries are an important consideration in this costing study. While all of the measures of financial performance indicated that there is a strong business case for Web-based training, financial indicators suggest that the amount of multimedia content in a course is the most significant factor in cost. The TCP/IP course, which had the least cost savings and a breakeven of 112 students, contained a five- minute video segment. As a result, the number of hours spent in multimedia production was far higher than for the other courses. The synchronous routing course, which was the most cost-effective course in the pilot, with a breakeven of four students, contained only a few graphics and live audio. The limited amount of multimedia content in the synchronous course offset the higher costs of course delivery resulting from the cost of having a live instructor present during delivery and the greater student salary costs due to the extra time required to deliver the course. In comparing the number of hours spent on multimedia development across all courses, the variations are relatively small. This indicates that all of the elements discussed previously are important to consider when costing Web-based course development. The methodology used in this case study can, therefore, be used in future cost-benefit studies of Web-based training. The American Journal of Distance Education, Vol. 13 No. 1, 1999, pp. 25-43 Note The research for this paper was supported in part by Industry Canada's Network of Centres of Excellence in Telelearning. The authors wish to express their appreciation for the comments of the anonymous reviewers on the first draft of this paper. References American Society for Training and Development (ASTD). 1998. Bibliography: Evaluation. Alexandria, VA: ASTD. Available online at . American Society for Training and Development (ASTD). 1998. Bibliography: Return-on-investment. Alexandria, VA: ASTD. Available online at . Aub6, S., M. Chilibeck, and D. Wright. 1996. Cost analysis of video conferencing for distance education at the University of Ottawa. Working Paper, 96-45. Ottawa, Canada: Faculty of Administration, University of Ottawa. Bates, A. W. 1995. Technology, open leaming and distance education. London: Routledge. Hall, B. 1997. Web-based training: A cookbook. New York: John Wiley & Sons. Hall B. 1998. The Cost of Custom WBT. Inside Technology Training July-August, 46-47. Higher Education Information Resources Alliance. 1994. What presidents need to know about the payoff on the information technology investment. Background papers for HEIRAlliance, Executive Strategies Report #4. Boulder, CO: CAUSE. Russell, T. 1998. The "No Significant Difference" phenomenon as reported in 248 research reports, summaries, and papers, 4th ed. Raleigh, NC: North Carolina State University. Trevor-Deutsch, L., and W. Baker. 1997. Cost/benefit review of the interactive learning connection, University Space Network Pilot. Ottawa, Canada: Strathmere Associates International Ltd. Introduction Costing Methodology Table 4. TCP/IP Course (Asynchronous): Cost and Time per Developer Task Table 5. Routing Course (Synchronous): Cost and Time per Developer Task