RESEARCH ARTICLES

A Comparison of the Acceptability and Effectiveness of Two Methods
of Distance Education: CD-ROM and Audio Teleconferencing

James E. De Muth, PhD, and Ruth H. Bruskiewitz, MS

University of Wisconsin-Madison

Submitted February 9, 2005; accepted April 22, 2005; published February 22, 2006.

Objectives. To fulfill a need for convenient and effective continuing education for pharmacists
throughout Wisconsin by developing a pharmacy continuing education program using 2 different
methods for distance education: audio teleconferencing and CD-ROM.
Methods. Eighty pharmacists were recruited from a 2003 University of Wisconsin distance education
program, 47 of whom participated in the course using the traditional audio teleconference method, and
33 of whom participated using a home study CD-ROM containing the same material presented in the
teleconference. Volunteers were required to complete a pretest, a first posttest immediately following
completion of the continuing education course, a second posttest 1 month following the conclusion of
the course, and an evaluation.
Results. The CD-ROMs were a more acceptable method for distance education than audio telecon-
ferencing and resulted in better retention of the course information.
Conclusion. These study results can help coordinators of continuing pharmaceutical education programs
to determine the most effective and acceptable method for future distance-education opportunities.

Keywords: distance education, continuing education, audio teleconferencing, CD-ROM instruction

INTRODUCTION
From 1968 to 2001, Extension Services in Pharmacy

at the University of Wisconsin-Madison School of Phar-
macy used the Educational Teleconference Network
(ETN) to provide distance-education experiences for
pharmacists in Wisconsin. ETN utilized an amplified
telephone signal to share information among groups of
individuals assembled at various distant locations
throughout the State.

1 ETN provided a relatively cost-
effective and efficient method for the delivery of continu-
ing education (CE) to an average of 421 pharmacists each
year for the annual fall pharmacy CE offering, at an av-
erage of 72 sites throughout Wisconsin. These averages
are basedonenrollments andsites fromthelast 15years of
ETN programming. These annual enrollments repre-
sented 15 % to 20% of the registered pharmacists in Wis-
consin. Advantages of ETN included: (1) convenience for
pharmacists; (2) a relatively inexpensive delivery method
for program developers: (3) a flexible means for reaching
sites in states adjacent to Wisconsin; and (4) the availabil-
ity of at least one listening location in each of the 72
counties in Wisconsin. Each Tuesday evening during

the 8-week program, a prerecorded lecture was broad-
cast on a designated phone line. An open microphone,
live question-and-answer session followed each broad-
cast. Pharmacists were able to access the annual program
over the ETN system; most from within 10 miles of their
home or workplace. Due to increased costs to maintain
the system and decreased usage by other University
departments, the State abandoned the ETN system in
June 2002.

The loss of the ETN system has provided Extension
Services in Pharmacy with a unique opportunity to fulfill
its mission to provide CE opportunities to pharmacists in
Wisconsin regardless of their proximity to a listening lo-
cation. Live programs held in Madison and Milwaukee
continue to serve the areas of the state where pharmacists
are concentrated; however, ETN traditionally provided
pharmacy continuing education for the remainder of the
State. Thus, alternative methods of delivery to distant
portions of Wisconsin were sought.

2002 Fall Teleconference Study
To continue offering the teleconference format for

pharmacists after the abandonment of the ETN system,
Extension Services in Pharmacy initiated a conference
call, using a State-operated system (WisLine), as an al-
ternative delivery system in fall 2002. This delivery sys-
tem was similar to the previous teleconference system and

Corresponding Author: James E. De Muth, PhD, University
of Wisconsin School of Pharmacy, 777 Highland Ave.,
Madison, WI 53705, Tel: 608-262-2422, Fax: 608-262-2431.
E-mail: jedemuth@pharmacy.wisc.edu

American Journal of Pharmaceutical Education 2006; 70 (1) Article 11.

1



making the changes virtually transparent to pharmacist
participants. The 8-lecture broadcast, ‘‘An Update on
Drug Interactions,’’ was originally planned for 72 listen-
ing locations (1 per county); however, due to budget con-
straints and security issues that required an aide to be
present at each listening location, most of the county sites
were not available for evening broadcasts. To compensate
for the loss of these public sites, several private sites, in-
cluding hospitals, were recruited to provide listening
locations. At the time of the first WisLine broadcast in
September 2002, 16 public sites (out of the original 72
county sites) and 16 private sites were available.

To help meet the needs of Wisconsin pharmacists un-
able or unwilling to attend one of the 32 available sites,
a CD-ROM was produced containing the same course
material as that broadcast over the WisLine system. The
CD-ROM contained PowerPoint slides with a voice-over
lecture track for each presentation. Because the course
was originally marketed with the option of attending at
one of the original 72 listening locations, 67 pharmacists
outside a reasonable driving distance of the 32 remaining
available listening sites were offered the option of either
a full refund or the CD-ROM version of the course. With-
out exception, all of those pharmacists chose to take the
course using the CD-ROM method rather than obtain a re-
fund. A CD-ROM, course workbook, final examination,
and course evaluation were mailed to 67 pharmacists prior
to the first broadcast. Of the 350 pharmacists who origi-
nally registered for the WisLine program, 283 continued
to participate via the audio teleconference, using course
workbooks containing copies of the speakers’ slides and
alivequestion-and-answersessionfollowingeachlecture.

CD-ROM technology, similar to online educational
programming, offers the advantage of participation at any
time and any location (ie, at work or at home). Also, like
traditional home study courses, online or CD-ROM pro-
gramming can be completed at a pace that is comfortable
for the individual learner. Numerous studies have looked
at online instructional materials versus face-to-face in-
structional programs and the findings may be applicable
to CD-ROM programs verses live teleconferences. Stud-
ies comparingface-to-face instructionand onlinelearning
have found no significant differences in effectiveness of
learning for college students and younger learners.

2-5

Other studies have evaluated the benefits of digital in-
struction for health professionals, including effectiveness
as measured by changes in pretest and posttest measure-
ments.6-10 However, only 1 study could be identified that
compared face-to-face instruction with computer-based
training for practicing health professionals.11 In that par-
ticular study, there were no significant differences in
either students’ acceptance of or the effectiveness of the

2 methods. No studies were found that compared audio
teleconferencing with asynchronous digital formats.

In a separate study of pharmacists participating in the
2002 program, CD-ROM technology was compared with
the audio teleconference system broadcast.

12 Course
evaluationsfor pharmacistsparticipatingin the traditional
audio-teleconference program (n 5 212) were compared
with those pharmacists opting for the same course mate-
rials on CD-ROM (n 5 67). For both groups, the average
response rate was 82%. Teleconferencing and CD-ROM
technologies were rated the same on the majority of the
evaluation questions. Differences identified included: (1)
the audio quality was rated higher on the CD-ROMs than
for the teleconference, which involved telephone line
transmission (p , 0.05); (2) the lectures recorded on
CD-ROM were rated as more effective than the lectures
via audio teleconference (p , 0.005); and (3) the same
course content presented on CD-ROM was perceived as
more beneficial than by teleconference (p , 0.005). In
contrast, the pharmacists’ median response when asked
the value of using 2-way audio teleconferencing for ques-
tion-and-answer sessions to clarify points of the lecture
(not available to CD-ROM learners) was ‘‘uncertain’’.

2003 Fall Teleconference Study
Based ontheresultsofboth thecourseevaluationsand

the previously reported 2002 study, along with the expe-
riences of using this dual system of providing continuing
education to Wisconsin pharmacists, it was decided that
the same 2 methods would be offered for the 2003 fall
program. However, in advance of marketing the program,
listening sites were recruited throughout Wisconsin and
a CD-ROM program was immediately prepared for those
unable or unwilling to attend the teleconference sites. The
course brochure offered pharmacists the choice of either
attending the traditional teleconference or completing the
CD-ROM program, regardless of their geographical loca-
tion or proximity to a listening site.

The 2003 course, A Review of Selected New Drugs in
the Beginning of the 21st Century, continued the tradi-
tional format of 8 lectures. Both groups of pharmacists
(CD-ROM and teleconference) had access to the live
question-and-answer session. Pharmacists receiving the
CD-ROM format were given a toll free number and ap-
proximate times to call, enabling them to join the telecon-
ference participants for each of the question-and-answer
sessions immediately following each of the 8 lectures.
Extension Services in Pharmacy successfully recruited
and made available to pharmacists a total of 37 potential
listening locations. When the course began, pharmacists
participated at 28 listening locations and attendance at
these locations ranged from 2 to 13 pharmacists.

American Journal of Pharmaceutical Education 2006; 70 (1) Article 11.

2



In addition to the presentation of the same course
material using 2 different delivery systems, a more rigor-
ous study was designed and executed to evaluate not only
the acceptability of CD-ROM home study courses as
compared to audio teleconferencing (similar to the 2002
study), but also the effectiveness of the 2 delivery meth-
ods in improvingpharmacists’ knowledge.The objectives
of this study were to: (1) develop and present a pharmacy
continuing education program using 2 different methods
for distance education; (2) evaluate the acceptability of
the 2 delivery systems; (3) measure and compare the
amount of cognitive gain using the 2 delivery methods;
and (4) examine the impact of selected demographics on
acceptability and cognitive changes.

METHODS
The identical pharmacy continuing education pro-

gram was offered using either a home study CD-ROM
(with optional access to the live question-and-answer par-
ticipation) or an audio teleconference program with a live
question-and-answer session following each lecture. Par-
ticipation in the live question-and-answer session was
also optional to teleconference participants since they
could leave the location prior to the start of the ques-
tion-and-answer session. All course participants were re-
quired to complete a posttest and an evaluation form.
Additional requirements for study volunteers were the
completion of a pretest and second posttest. Pharmacists
who registered for the course were asked to volunteer for
the study and were offered a 15% discount on course
tuition as an incentive to volunteer. The study was sub-
mitted to the University of Wisconsin Health Sciences
InstitutionalReviewBoard,whereitreceivedexemptstatus.

To measure cognitive change, pharmacists were pre-
sented with 2 parallel examinations of course materials
before and after participation in the course (a pretest-
posttest design with one of the parallel tests serving as
the pretest and the second parallel test serving as the post-
test). Development of the 2 examinations consisted of
obtaining 10 questions from each lecturer, for a total of
80 questions. These questions were administered to a vol-
unteer group of 9 Madison hospital pharmacists who did
not register for the course; the difficulty of each question
wasevaluatedbytestingthesevolunteers.The2easiestand
the 2 most difficult questions for each lecture were elimi-
nated and the remaining 6 questions per lecture were ran-
domly divided into the pretest and posttest questions,
resulting in at least 3 questions for each lecture. Two ques-
tions were randomly selected from the 16 most difficult
questions to complete the 25-question pretest and posttest.

The extent of each pharmacist’s participation in the
program was taken into consideration. Only pharmacists

reporting participation in a minimum of 7 lectures were
included in the study. The volunteer pharmacists were
classified into 2 categories based on course format partic-
ipation: audio teleconferencing or CD-ROM. Assessment
of the pharmacist’s satisfaction with both the delivery
system and course content required that a course evalua-
tion form be completed and returned at the end of the
program. Pharmacists’ acceptance of each delivery sys-
tem was assessed using a series of 5-point Likert scale
questions. Effectiveness of the 2 delivery systems was
assessed by measuring change in pharmacists’ knowledge
(posttest vs. pretest scores). Figure 1 illustrates the time-
line for the study methodology.

Data were analyzed using Minitab software (Release
12, Minitab Inc., State College, Penn). In most cases, the
evaluation results reported in the tables are presented as
means, for ease of comparison. However, since the eval-
uation statements were scored on a 5-point ordinal scale,
statistical analysis required the use of non-parametric pro-
cedures, primarily the Mann-Whitney U and Kruskal-
Wallis tests. For cognitive test results, parametric 2-sided
t tests and analysis of variance tests were used. Where chi-
square tests were performed on a 2 x 2 contingency table,
the Yate’s correction for continuity was used.

13 Unless
otherwise specified, all tests were performed with a 95%
level of confidence (p # 0.05).

RESULTS
A total of 297 pharmacists and other health profes-

sionals enrolled in the course: 172 chose the audio tele-
conference and 125 participated via CD-ROM. There
were 270 evaluations returned (156 from teleconference
participants and 114 from CD-ROM users), representing
a total response rate of 90.9%.

Since study participants volunteered to take an addi-
tionalexamination(thepretest),thefirstinquirywastode-
terminewhetherthestudyparticipantsdifferedsignificantly

Figure 1. Timeline of Study Methodology.

American Journal of Pharmaceutical Education 2006; 70 (1) Article 11.

3



from the non-study participants with respect to demo-
graphics or evaluation results. No significant differences
were found based on the demographic information
obtained on the evaluation form: academic degree, bach-
elor of science (BS) vs. doctor of pharmacy (PharmD) or
advanced degree (p 5 0.912); years of pharmacy practice
(p 5 0.700); practice setting, institutional vs. retail (p 5
0.466); practice position, management vs. staff (p 5
0.219); for teleconference participants, the distance trav-
eled to listening location, above vs. below the median
miles (p 5 0.276); and for CD-ROM participants, pre-
vious participation in an ETN program (p 5 0.208). In
addition, there were no differences in responses to all 15
evaluation statements discussed below (p values varied
from 0.063 to 0.97). Thus, for analysis of method accept-
ability, all of the usable 270 evaluations were considered
for the 2 delivery methods (course evaluations) and only
the study participants (n 5 80) were used for evaluating
the effectiveness (test results) of the CD-ROM method
compared to the teleconference format.

Demographics
The majority of the pharmacists participating in the

2003 distance education experience had BS degrees
(89.3%), held staff positions (73.3%), were from insti-
tutional settings (61.8%), and had 20 or more years of
practice experience (60.8%). For those choosing the tele-
conference option, the average roundtrip traveling dis-
tance to the 28 listening locations was 14.1 miles
(ranging from less than 1 mile to 100 miles roundtrip each

night). These variables were evaluated to determine any
relationships among the demographics. There was a sig-
nificant relationship between academic degree and years
of practice experience (x

2
5 33.03, p , 0.001), with

51.8% of the advanced degrees held by pharmacists with
10 or less years of experience. All other comparisons
among the various demographic variables showed statis-
tical independence. Also, the vast majority (92.0%) of
those choosing the CD-ROM option had previously par-
ticipated in at least 1 audio teleconference program. Table
1 compares the demographics for those who chose the
CD-ROM delivery method with those who chose telecon-
ference delivery method. Demographically, there were no
significant differences between the 2 volunteer groups
based on the delivery system chosen, except for the num-
ber of years of professional practice. Pharmacists choos-
ing the CD-ROM format had significantly fewer years of
practice thantheirmoresenior counterpartswhochose the
audio teleconference (Table 1).

From a list of possible motives, participants were
asked to choose their top 3 reasons for participating in
the distance-education program. Results were weighted
by 3 for the first choice, 2 for the second choice, and 1 for
the third choice. Based on pharmacists’ top 3 reasons for
attending continuing education programs, the most pop-
ular reason was to enhance their general pharmacy knowl-
edge. Of the 252 pharmacists responding to this section of
the survey, 97.2% chose enhancing knowledge as one of
the top 3 reasons for participating in continuing educa-
tion. Fulfillment of continuing education requirements

Table 1. Comparison of Demographics Between Pharmacists Participating in a Continuing Education Course Delivered by
Teleconference and CD-ROM

Teleconference, No. (%) CD-ROM, No. (%) P*

Highest academic degree

BS 132 (92.3) 93 (85.3) 0.076

Advanced Degree (MS, PharmD, etc) 11 (7.7) 16 (14.7)

Total years of pharmacy practice

10 years or less 18 (12.5) 21 (19.8) 0.066

11-20 years 31 (21.5) 28 (26.4)

20-30 years 53 (36.8) 40 (37.7)

311 years 42 (29.2) 17 (16.0)

Years - Mean (6SD) 25.3 (10.2) 20.9 (10.7) 0.001

Employment setting

Institutional 83 (63.4) 56 (59.6) 0.565

Retail 48 (36.6) 38 (40.4)

Employment position

Management 37 (28.2) 23 (24.5) 0.528

Staff 94 (71.8) 71 (75.5)

*Statistical tests used were 2-sample t test for continuous outcome (years) and chi-square test for discrete outcomes (with Yate’s correction for
2 x 2 designs)

American Journal of Pharmaceutical Education 2006; 70 (1) Article 11.

4



(83.7%) was second among the top 3 reasons. Education
as part of their professional responsibilities (73.8%) was
third, followed by gathering information in more depth or
detail (25.8%), networking opportunities (9.1%), getting
help with a specific problem (2.7%), and development of
a specialty pharmacy practice (2.7%). With respect to the
order of importance, these results were similar to the find-
ings in a similar study of Wisconsin pharmacists and
their attitudes toward the availability and acceptability
of various distance-education delivery systems.

14 When
reasons for attending responses were divided by the
method of delivery chosen, the results were almost iden-
tical between the 2 groups. One obvious difference was
that no one who enrolled in the CD-ROM version chose
‘‘network and/or meet with colleagues’’ as a reason for
participating, whereas this response represented the fifth
most popular reason for teleconference participants.

Evaluation of the Presenters
As measured over the last 5 years, there has been little

variability in the pharmacists’ opinions of the overall
value and effectiveness of the speakers for those partici-
pating in the audio teleconference (Table 2). There was
more variability with respect to the depth of the presenta-
tions (F 5 3.30, p 5 0.04), but there were no significant
differences with respect to the value of the presentations
(an overall median response of 4 5 ‘‘valuable’’) and the
effectiveness of the lecturers to present their course mate-
rials (overall median response of 4 5 ‘‘effective’’). As
mentioned previously, in 2002 the pharmacists felt that
the lecturers were more effective using CD-ROMs (p ,
0.005). However, in 2003 there were no significant differ-
ences between the teleconference and CD-ROM respond-
ents with respect to the composite evaluations for all

8 speakers, even though the mean response was slightly
higher for the CD-ROM participants (p 5 0.33). There
was no significant difference between teleconference
and CD-ROM participants with respect to their evaluation
of either the value of the presentations (p 5 0.18) or the
depth of the presentations (p 5 0.26). Due to variability in
speaker style and each speaker’s comfort with distance
education, individual lecturers were not evaluated in this
study;onlyacompositescoreforall8lecturersisreported.

Acceptability and Effectivness
Table 3 summarizes participants’ responses to a series

of 15 evaluation questions. Most were general questions
forbothdeliverysystems,butsomewerespecificforeither
the teleconference or CD-ROM method. For this study,
effectiveness was measured by evaluating the change in
the pharmacists’ scores (posttest vs. pretest scores). Vol-
unteers were required to complete and return the pretest
prior to receiving any of the course materials to prevent the
review of course materials prior to taking the pretest. Par-
ticipants were required to take the posttest after complet-
ing the course. However, when the posttest was returned
for grading, it was discovered that an error had been made
in preparing the posttest materials. The pretest and postt-
ests both contained the same test questions, instead of the
posttest containing a parallel set of test questions. The
study investigators decided to mail the volunteers a second
posttest (consisting of the alternate set of questions) 1
month following the conclusion of the course. Since many
pharmacists completed the final examination as they lis-
tened to the lecture (via either teleconference or CD-
ROM), often treating it as an open-book examination, it
was felt that this second posttest might actually provide
a better assessment of the pharmacists’ learning. As seen
in Table 4, results from the later test (labeled ‘‘Second
Posttest’’) were slightly lower than the immediate final
examination results (labeled ‘‘First Posttest’’).

Limitations
The primary limitation of the study was the use of self-

selecting subjects based on either convenience or personal
learning styles. Ideally, volunteers would be randomized
intothese2typesofdistance-educationprograms.Asecond
limitation is the unequal samplesizes, with more volunteers
participating in the teleconference format. Appropriate sta-
tistics were used to adjust for unequal sample sizes.

DISCUSSION
Acceptability of the Methods

Although not statistically significant (Table 3), the
overall course evaluation ratings given by pharmacists
in the CD-ROM group were higher (median of 5 5 ‘‘very

Table 2. Comparison of Five Years of Teleconference
Speaker Evaluations

Mean Response (SD)*

Year n Value Effectiveness Depth

1999 457 4.12 (0.68) 3.89 (0.77) 3.37 (0.5)

2000 389 4.14 (0.81) 3.92 (0.86) 3.33 (0.7)

2001 347 4.05 (0.74) 3.92 (0.73) 3.39 (0.54)

2002 244 3.98 (0.86) 3.80 (0.88) 3.49 (0.58)

2003 156 4.03 (0.85) 3.89 (0.84) 3.46 (0.73)

Statistical
Comparisony

F 5 2.21
p 5 0.11

F 5 1.01
p 5 0.37

F 5 3.30
p , 0.04

Based on 5-point Likert scales: value (1 5 not valuable to 5 5 very
valuable), effectiveness (1 5 not very effective to 5 5 very effec-
tive), and depth (1 5 too shallow, 3 5 about right, 5 5 too deep)
yIdeally, a non-parametric test would have been performed on this
information, but original data were not available; only previously
reported means and standard deviations were available

American Journal of Pharmaceutical Education 2006; 70 (1) Article 11.

5



satisfied’’) than those given by pharmacists in the telecon-
ference group (median of 4 5 ‘‘satisfied’’) with respect
to: (1) the subject matter of the course; (2) the value of
the handout materials to complement the lectures (both
groups received the same workbooks with copies of the
speakers’ slides); and (3) the audio quality of each
method. With respect to the audio quality, although the
mean rating was lower for the CD-ROM, the median rat-
ing was higher. Almost 10% of CD-ROM users were
dissatisfied with the audio quality of the CD-ROMs, com-
pared to only 2% of teleconference participants who were
dissatisfied with the audio quality of the teleconference.
Poor audio quality was also the major complaint made by
the CD-ROM study group. It is possible that one batch of
CD-ROMs was produced at a lower audio quality and this
should be monitored more closely for future programs.

Specific evaluation questions designed for CD-ROM
users found that the video quality of the CD-ROM and the
user friendly nature of the CD-ROM program were ‘‘very
satisfactory’’ (each median 5 5). At the same time,
attendees at the 28 sites were ‘‘satisfied’’ (median 5 4)
with the physical facilities at their listening locations.

They were also ‘‘satisfied’’ with the value of the ques-
tion-and-answer sessions in clarifying lecture points or
providing additional information related to the course.
Only 24 of the CD-ROM course participants indicated
that they took advantage of the optional live question-
and-answer sessions following each lecture. Those par-
ticipating in the sessions provided a median response of 4,
indicating a similar level of satisfaction with the question-
and-answer sessions among both CD-ROM users and
teleconference participants.

Since the 2 groups (teleconference and CD-ROM) of
users were similar with respect to demographics (Table
1), it was not surprising to find similar responses (medians
of 4 5 ‘‘agree’’) when asked if the material covered in the
course was directly applicable to their professional prac-
tice. Also, both groups considered the course material to
be fair/balanced and not commercial in nature (medians
of 5 5 ‘‘strongly agree’’).

However, there were several significant differences
in the pharmacists’ responses to other questions that were
related to thedelivery methodused in thestudy.CD-ROM
participants (1) were more likely to find the course ‘‘very

Table 3. Response to Evaluations Completed by Participants in a Pharmacy Continuing Education Course

Teleconference,
Mean (SD)

CD-ROM,
Mean (SD) P*

Responses based on a 5-point Likert Scale (5 5 very satisfied, 1 5 very dissatisfied)

The subject matter of the course 4.34 (0.59) 4.47 (0.63) 0.093

The value of the handout material to complement the lectures 4.40 (0.64) 4.40 (0.75) 0.990

Audio quality 4.30 (0.68) 4.24 (1.03) 0.620

Video quality of the CD-ROM — 4.60 (0.51) —
The CD-ROM program was user friendly — 4.75 (0.47) —
The physical facilities at your listening location 4.41 (0.64) — —
The technique as a means of obtaining ‘‘in depth’’ continuing education 4.26 (0.71) 4.59 (0.59) ,0.001

The value of the question session in clarifying certain points in the
lecture and providing further information

3.98 (0.73) — —

Responses based on a 5-point Likert Scale (5 5 strongly agree, 1 5 strongly disagree)

Overall, I found this teleconference (or CD-ROM) course
to be very worthwhile

4.29 (0.61) 4.61 (0.52) ,0.001

As a result of this program, I have a greater understanding of the
some of the significant newer drug therapies

4.27 (0.50) 4.46 (0.50) 0.004

The material covered in this course was directly applicable to my
professional practice

4.16 (0.65) 4.20 (0.70) 0.514

The program was presented as a fair and balanced discussion and
was not commercial in nature

4.55 (0.51) 4.64 (0.50) 0.143

The stated objectives for this course were achieved 4.27 (0.55) 4.47 (0.55) 0.002

My personal objectives in attending the course were fulfilled 4.26 (0.58) 4.48 (0.55) 0.002

I enjoyed the CD-ROM technology, but would have preferred
to attend the live teleconference program. (for those attending
a previous ETN teleconference)

— 2.17 (0.90) —

*Results are reported as mean 6 SD for purposes of comparisons, but statistical evaluation involved the Mann-Whitney U test since the Likert
scales represented ordinal scales. Data reported in the text refers to the median responses used in the Mann-Whitney test

American Journal of Pharmaceutical Education 2006; 70 (1) Article 11.

6



worthwhile’’; (2) felt they had a greater understanding of
the significant new therapies discussed in the course; and
(3) had a greater sense that their personal objectives were
fulfilled. For all 3 questions, the median response from
CD-ROMuserswas‘‘stronglyagree,’’comparedtotheme-
dian response from teleconference users of ‘‘agree’’. Although
the response from the CD-ROM groups was statistically
significant, both groups gave a median response of ‘‘agree’’
to the question about the course achieving the stated
objectives. Also, when asked if the particular delivery
method used in the study was satisfactory as a means
for obtaining ‘‘in depth’’ continuing education, the CD-
ROM users provided a median response of ‘‘very satis-
fied,’’ whereas their audio teleconference counterparts’
responded they were ‘‘satisfied’’.

Finally, 82 of the CD-ROM users (72%) indicated
that they had previously participated in at least 1 audio
teleconference program. When asked if they would have
preferred to have taken the teleconference version of the
course instead of the CD-ROM version, only 4.8% agreed
or strongly agreed with the statement. Conversely, 67.0%
disagreed or strongly disagreed, indicating a preference
for the CD-ROM format.

Effectiveness of the Methods
As seen in Table 4, there were no significant differ-

ences in any of the test scores when comparing the 2 de-
livery systems (teleconference vs. CD-ROM). The pretest
scores for both groups were almost identical, averaging
around 45% (p 5 0.76). Correspondingly, the 2 groups
showed similar average scores on the first posttest (p 5
0.28) and the second posttest (p 5 0.065). Although not
statistically significant, the pharmacists using the CD-
ROM seemed to retain the course materials better than
their colleagues receiving the teleconference, as evi-

denced by the second posttest scores of 83.0% and
77.9%, respectively.

Significant differences were observed when the in-
dividual pharmacists’ scores were compared among the
pretest and 2 posttests. All pharmacists had a significant
increase in the knowledge scores on the first posttest, with
teleconference pharmacists increasing 47.8% (p , 0.001)
and CD-ROM pharmacists increasing 49.9% (p , 0.001).
As mentioned earlier, the second posttest may have been
a better measure of cognitive change because it was not
available for pharmacists during the presentations, having
been mailed after the first posttest was received from each
group. On the second posttest, there was still significant
improvement in performance over the pretest by both
groups (p , 0.001), but the CD-ROM pharmacists showed
an average 38.4% increase on their scores, while telecon-
ference pharmacists showed a 32.5% improvement. Al-
though the difference was not statistically significant, the
scores suggest that the CD-ROM format was more effec-
tive than the teleconference format in terms of partici-
pants retaining information presented in continuing
pharmacy education programs.

CONCLUSION
The use of either audio teleconferencing or CD-ROM

technology appears to offer acceptable and effective
methods for providing continuing education to pharma-
cists over a wide geographical area. Based on evaluation
results, the CD-ROM users tended to be more satisfied
withthedeliverymethodforobtainingin-depthinformation.
Also, they expressed greater satisfaction with the course
meeting itsstated objectives, as well as fulfilling theirper-
sonalobjectivesinparticipatinginthecourse.Bothmethods
were equally effective in increasing knowledge, as mea-
sured by differences in the pretest and posttest scores.

Table 4. Results on Examinations by Teleconference or CD-ROM Participation*

Teleconference
Participants,
(n 5 47)

CD-ROM
Participants,
(n 5 33) Significance

Pretest scoresy 45.5 (12.7) 44.6 (12.1) t 5 0.31, p 5 0.76

Posttest scoresy (immediately after program) 93.2 (6.37) 94.6 (4.34) t 5 1.09, p 5 0.28

Second posttest scoresy(one month later) 77.9 (13.1) 83.0 (11.29) t 5 1.88, p 5 0.065

Changes in test scoresz

Posttest - pretest 147.8 (14.6) 149.9 (12.3)

t 5 22.4, p , 0.001 t 5 23.2, p , 0.001

Second posttest - pretest 132.5 (15.0) 138.4 (13.4)

t 5 14.8, p , 0.001 t 5 16.5, p , 0.001

*Results reported as mean 6 SD
y2-sample t tests comparing teleconference and CD-ROM participants
zPaired t tests comparing individual posttest-pretest scores

American Journal of Pharmaceutical Education 2006; 70 (1) Article 11.

7



These study results can help course coordinators de-
termine the most effective and acceptable method for
future distance-education opportunities. Although Wis-
consin pharmacists were studied, the results of comparing
2 methods of distance-education delivery can be applied
by adult educators nation-wide. This information can
assist universities choosing among distance-education
delivery systems for nontraditional undergraduate in-
struction or continuing education programs.

REFERENCES
1. Parker LA, Monson MK. More Than Meets the Eye: The
Research on the Effectiveness of Broadcast Radio, SCA and Tele-
conferencing for Instruction. Madison, WI: Instructional Communi-
cation Systems, University of Wisconsin-Extension: 1980:32-3.
2. Arbaugh JB. Virtual classroom versus physical classroom: An
exploratory comparison of class discussion patterns and student
learning in an asynchronous internet-based MBA course. J Manage
Educ. 2000;24:207-27.
3. Diaz D, Cartnal R. Student’s learning styles in two classes: online
distance education and equivalent on-campus. Colleg Teach.
1999;47:130-5.
4. Neuhauser C. Learning style and effectiveness of online
and face-to-face instruction. Am J Dist Educ. 2002;16:99-113.
5. Aragon SR, Johnson SD, Shaik N. The influence of learning
styles preferences on student success in online versus face-to-face
environments. Am J Dist Educ. 2002;16:227-44.

6. Neafsey PJ. Computer-assisted instruction for home study: a new
venture for continuing education programs in nursing. J Contin
Educ Nurs. 1997;28:164-72.
7. Neafsey PJ. Immediate and enduring changes in knowledge and
self efficacy in APNs following computer-assisted home study
of the pharmacology of alcohol. J Contin Educ Nurs. 1998;29:
173-81.
8. Casebeer L, Kristofco R, Strasser S, et al. Standardizing evaluation
of online continuing medical education: physician knowledge,
attitudes, and reflection on practice. J Contin Educ Health Prof.
2004;24:68-75.
9. Curran VR, Hoekman T, Gulliver W, Landells I, Hatcher L. Web-
based continuing medical education (II); evaluation of computer-
mediated continuing medical education. J Contin Educ Health Prof.
2000;20:106-19.
10. Francis B, Mauriello SM, Phillips C, Englebardt S, Grayden SK.
Assessment of online continuing dental education in North
Carolina. J Contin Educ Health Prof. 2000;20:76-84.
11. Harrington SS, Walker BL. A comparison of computer-based
and instructor-led training for long-term care staff. J Contin Educ
Nurs. 2002;33:39-45.
12. De Muth JE, Bruskiewitz R. Comparison of two methods of
distance education for continuing education: Audio teleconferencing
and CD-ROMs. Am J Pharm Educ. 2003;67:Article 100.
13. De Muth JE. Basic Statistics and Pharmaceutical Statistical
Applications. New York, NY: Marcel Dekker; 1999:394-6.
14. Bruskiewitz R, De Muth JE. Availability and acceptability of
distance education delivery systems for continuing pharmaceutical
education. Am J Pharm Educ. 2005;69:Article 25.

American Journal of Pharmaceutical Education 2006; 70 (1) Article 11.

8