ED 188 561

yITHOP
TITLE

INSTITUTfON

'PUB DATE
NOTE

EDPS PRICE
DEKFTPTOPS

IDENTIFIrDS

DOCOMPIT RiSTIE

11\.0.12 922 '

Brown, (54.ep);en: Nathenson, Michael. u
Designing'Instructional Mmterlals: Guesswork or.
*Facts?'
Open Univ., Waltbt-, Blrchley, Bucks (En4landl:.Irst.
of Educational TechnOl^ay.
Nov '78*

'381). ,

1'MF01/PCO2 Plus Postage.
*Academic Ability: *Basle-Ski/11.s: *Externalregree
Programs: Foreian.Countri_es: tigher"Educatior:
*Instructional Ma*.erialsi Instructional"Sytems:
Introductory .C"urses: Language Skills: Mathematics
Anxiety: *Science Programs:-*Social Sciences:
Techni.9. Education
*Open UniversityjGreat Prit in)

ABSTPACT
A systematic study is reported,of the. matheuatica l

. skills and learning abilities of new students imdediately prier to
eptry int^ the Open University's (Ftl.glmrd), foundation courses in
t.echnology and social sciences. q'te intent was to provide predicve
informatiu about potential student' entry skfl.ls fo the team
preparing a newtechnology' foundation course. Basi langpage and

ematical skills were assessed by a guePtionna rt-test given to"
le of applicants for the courses. was foI# that ftathemAtical

skills were an area needing speciattention, ar chances ir course
materials mnd ..schedules*are propoSe It;:. assist südents in developing
Ole skills In the coutse. 1110.5 concluded that suph- systemaUc
evaluations cf critical rearner skills are needed as a preregbisite
to the desi6n of instructional material for ether courses, both at
the Open Univeraity and in other institutions. IMSE)

e

1

4
************

Peprcductions supplsi*.ed by EDP! are.the best that cah be mad? *

irt9p the orIgInal document. *

. .
,

-



0

If

co'

a

1.

ASSTRACT

a.

<,

S.

This is a reportoof a systematic study of the

mathematical skills and learning capabilities of new

Students immediately prior to entry into.T100 and

D101. the Open University, Foundation.Courseft in

6

Technology and Social Sciences respectively. The

aim of the Study waskto pravide predictive

information about potential student.entry skills for

. the Course Team jareparing T101; 'the new Technology

Foundation Course.

The report describes the mettiod used in the study

and expliiins how it forma part of-an overalloptrategy

4
for: ensuring that the design of the instructiona). l

material to be used in T101 is based onothe known.
t

* legirilin-g capabilities and requirements og its likely

. students.. ---T-iifindings of the -Study..are reportedk

t 1
. briefly and the prOblims ana advantages of usisg-thi's- .

0 .
is

kind of approach to cuVriculum divelopment are

discussed.

I.

pa
1

3
%nil 1 5 1980,

*

a



4

Desionind Instructional Materials:

. Guesswork.or Facts?

0

..1

Stephen Brown.

and

MiChael rathenion

The Open University

No4'ember 1978*

;

Oa

4

4

q.

04



BACKGROUND TO THE nippy
4,

I/ 2Undergraduates of the Open.University are normally required
to take two foundation courses towards their .deree. Each

(#.

foundation course is considered to be the equivalent ofrsix

months full-time study or one year part-time. There are five

foundation courses, one for each of the following faculties:

.

Ar sl.Social Sciences, Science, Mathematics, Technology.

.14

/,

. A major task for foundatfon courses'is to bridge the gap

between school.and aniversity 14e1 study in terms of studentA4

study, communication and dialectical The Open University

0

was created to provide educational opportunities for people who

have missed out for some reason on the ediscatibn 'facilities

avaitlable to /them in earlier life. Consequently, Open.University

ig .

students come from a wide range of backgrounds, employment and
ONim

6
educ!ation, and they can differ widely on entry in terms of ages

e
0 k

.

, skills and know/edge. A distinctive feature Of foundation courses
,I 0

therefore must be their breadth Of appeal Snd their'ability to
4.S

meet the educationaLreAtirements of an extremely diverse student

: population.

T101 is the code nuMber of the new Open University Technology
p.

Folindation Course entitled "Living with.Technology". It is plandbd
.

to replace the'existing course (T.106: The Man-Made.World) in 1986.
0

TIO1 is a course about technology rather than in technolOgy in the

sense that it is concerned with the ways in which technology

influences both our present way of life and.our future. The team
4

A

1..f



c .. r /
4

.
i 77

. 'reapabible for producipg the course,
the Course TeaM, believes

.
.

t

. %
. .

that,Living with Ttchnolow will be of vital
interest and releva-nce

"

4

*

O
.

to-a broad spectrum of students, not
just those with an interest

in the engineering aspects of.tethnology.
Consequently, the Course

d

0
Team wants to piement the coursliin a way which will make it

accessible to studints, yhatever their previous background or
skills.

A basic and, on the face of it self-evident,
tenet of goodo

teaching is that the teaching 'material should be
matched:toethe

4?

akills and learning abilities of the students at
whom it is

directed. That is to say, the
teaching material should begin at a

5.

level which corresponds to the skills of
the students immediately

prior to taking the-course and it should-proceed
at pade

consonant willh thefr abilities to assimilate
the new*. material.

Despite the obvious common sense of
this approach it is not

always carried out in practive.fOr a very good reason.
Nkt-the

time the-teaching material is i,eing
prepared neither the entry.*-

skills nor the4earm1ng
cabililities of the future students *

known. a

In the case of T1 the.tar get student population
for the new

Technology FoundStion, Øourse is broader
than that currently

registered for T100. JTAble 1 shows that
only 11% of the

application' from ew stUdents.for fouriaation
courses'are for-2100.

ir

rf the Course Team is successfUl a much larger
proportion of

N
4

0

r
. students who currently apply for other

foundation courhea in the:
.

,

University (i.e., in the Social Science,
Sciere, Arts and

44

Mathematics Faculiies) will also apply
foripOl.



4?

I.

P

Table 1: Applications from nev7 students Pow Foundation Courses

1

averaged over197741979. (Note the pereentagefigures
f`eitt?

quoted are approximate only o.the nearest ini;eger.) -.xi.

. ...,. .
. ,

a

-Foundation
Course

Applications ,

Arts
.

. 34
,

Social
SCience

1

26

.

Science

-.4

, le,
,

,

Maths
; (

V.

14
.

..

Technologi
It

0

a
S. 0

4.

- S.

a.

*



I.

V.

'Detailed inforrtion on studpni abilities prior to course

commencement is not availSble. The only available'institutional

,research data.list student characteristics.such a a'ge, occupation,

sex,Ireviktus Oalifications, eter or describe student.performance

. on a partidular coursele.g., number of assignments passed, number
. .

,
oi.students talking the' exam, average exam scores, etc. What these

t
-4

kinds of data do not tell the course 1esigner4is. what skills tbe 4

students are likely to possess at tfie loeginning pi- the course, or

how theyrwillfespond to diffiiirent teaching strategies. In other
. .

. ,.

words, the existing survey data.are not sufficiently R5edictive.

4
To overcome these difficulties the Course Team has dedided to

adopt a three-livilet_strategy aimed at ensuring.that the course

material is tailored as closely as possible to'the learning

requirements of its studer4.
Pt.

. I .

The first part of the strategy is a skills survey aimed at
A

providing quantitativeoinformation about the likely entry skills

and learning abilit f ;rtol students: The intended function of

this 'information is to provide mime guidelines for setting

*stahdards in'an areS fraught with-dincertainty, conflictin0 opinions

and myths. ,

Thd'second part is ascheme for developmentally taistirig the
.

teachinsvmaterial for the first six stlidy week's of the course in

draft form. The first six study weeks are curcual to successful



.1

A

study' of the entire.cOurse since they will lay the foundations

of'the basic numeracy aild literacy skills in addition to

intr;ducing students to some iMportant technological activity*

areas. The results obtained from the developmental testing will

he used to ratfify or revise as necessary the findings from the
,

skills survey. 4

The third part of th6 st?htexy is
4

programme.with the aim of assessing the effectiveness of the.
Ito

f

,

teachtng fn the coursP.and rotrosppctively idehtifying learning

ong-term course evaluation

J

difficulties' encountered by the students. It is recognised that

the previous two approaches (a pre-course skills sUrverand,
f

dovekrpmemtal testing 'of parts of the course) although'useful for
"

eatablisbing reference points,,are unliRely in themselves'to .

enable the Course Team to produde'the best posflible cours.p on a

4
4

first attelipt.i/.It will bewnec essary to assess,the effectiveness

. .1

ofidge course as. a whole in its 'final po lished form-amd to\make
s

mpdifications0o-1. For this reason a Course EvalUation Team has

been establis4111 with the responsibiUty for collecting feedback
*-

from students on all arpects of the coursb during its firs year

presentation, .(4980) .

for modifying the co

The vesulting feedback data,will be.the basis

urse as necessary during the second yoar (1981),

amd for re-presenting a revised course irk the. third-year (1982).

It is hoped that fhe revised course will be mAched sufficiently

clqsdly to studentoe learning requirements for it to be able to

run substantially unchanged fol. six years..

In due course reports will hd.produced discupsin e.results

of the second and third parts of the Course Team strategy. This

I.

or'



V A
2

paw. is concernbd with the f4rE4 part: the kills Survey.

st,

0,
THE SURVEY 'METHOD

The aim of the survey

r

The two skills'of literacy and numeracy referred to. above were

identified as being of,importance to the study of technology. Of

these, n*meracy was felt to be a particularly important topic for

investiga0.; because the ability to solve problems numerically is

. .

considered to be one of the most important skills of technologists;

it is also a skill which has Caused difficu/ties for studentsAon

ihe Curr:ent Technology Foundation Course. Moreover, within the

Open Univasity as a whole, mathematics seem to be a stumbling

'hock for students in as much as courses with substantial mathematics
),

/ e
. have hidher withdrawal rates than courses with little or none. More

,
.

fundamental even than basic numeracy are the algorithmic skills

,,z.,.equird to .carry out the computations involved once the problems

have been appropriately xpressed. The aim of the survey was to

provide the Course.Team with detailed useful information about

students' entry level algorithmic Skillh and learning capabilities.

-

in areas relevant tosthe subject matter_proposed for the beginning

of the course:
41.

(1) Basic Arithmetic, including addition, subtraction,

multiplication and division, negative nuMbeirs, precedence'

of operators and brackets.

(2) Fractions and Ratios, including addition; subtraction4
0-

multiplication.and division of.fractiipns and.the cpnversion

of.fractionsto ratios and vice-versa.

f



(3) 'Percentages, including Calculating percentages,

converiing percentages to fractions and vice-versa and

percentages to decimals and vice-versa.

(4) Decimals, including converting fractions to decimals

and vice-versa, multiplication and division of decimals
,

and rounding, truncating and,signIficant figures:

4

(5) Areas ang Volumes, including calculating the areas of

rectangles, parallelograms, ttiangles and circles, the

Surface areas of simple solids,and the,volumes of simple

sonde.

(6) Graphs, including plotting and reading coordinates and

lirplotting graphs.

4
8

Withi

n1,1111,s

#
of these six areas the Course Team wanted answers

,

to four ques :- .
' e

(1) 'What Proportion of the sample targi population are able to

carry out simple calculations without any help from T101?

.-(2) What Torlioportion of the sample are able to perform the

calculations with oar a 'brief reminder'?

(3) What proportion of the samille required detailed teaching in

order to learn (or releaim) the algorithmic skills required

to perform the calculations?

(4)' How Much time Le spent studying the detailed teachinle /



-

Design of the Veationnaire, 4
I 1

A separate questionnaire was prepared for each of the six

areas and was divided into a number of blocks of calculations.

Each block covered three or more subskilis of the.general skill

being 'assessed.

For example, in the Baste Arithmetic questionnaire, the first !

block assessed simple addition, subtraction, multiplication, and

division; the second aAsessed the use of 'negative nuMbers1;. the

.third /precedence of operatorie; and the fourth blodk assessed the

ft

use of brackets.

Students were asked to tackle all the bloas of calculations at

the beginning of a questionnaire, then.to mark theit answers to the

first block.' If they got them all right they proceeded to mark

their answerr to the iecond block. If they- got any wrong. or were

-

uriable to do any of the calculations in the first block they were

directed to a brief reminder of how to do thosekinds of calculations,

then to another similar set of calculaiions to tadkle. If they got

all the talculations right &Ater the brief reminder they were asked

to proceed with marking their answers to the second block. Otherwise

they were asked to study some detailed teaahing and to attempt a

further,set ofcalculations. The detailed teaching.material used

was partly culled from a variety of existing OU and other sourees

and partly prepared espectally for the survey where none.of the

existing material seemed suitable. :After marking Ifieir answers

to the *third set of calculations ihey were asked to proceed with

marking their answers to the next block. This procedve, which was

reimated for eadh block in turn, is summarised in Figure 1.
#

1.4

f



.

Tackle all
Blticks

ve

04:; 0a

-

- 10 -
or

all
-righI

Read
-detailed
teaching

F.

z

Tackle Mark
more answer.s
probldms° ,

Mark answers
to Block 2

1111111/111M

yes

Fig- 1. Procedure for completing a' questioimaire
9

a



4

0

a
g Students weile also asked to record the amount of time they

spent studying the detailed teaching and answering the problems

which-follow,de it.

ik
SelectioWof the sampleepopuAtion

.

t

There was, at the tide of carrying out the survey, mo poo1 of
. A .

applicants for the course which could be sampled fort their pre-cburse
(

. 2
skills Or learnihg capabilities. Such a pool will not be available

until lat, 1979, i.e., when the course will almost readi for
V

presentatidn. Instead a sample poiulation had to be defined which

would be is representative as possible of the kindsi"of student T101
110.

is intended to attract. The target student pOpulation was

identified as the present range of students who apply for T100 plus

a larger proportion than'at present of studedts who are attracted

.to the Social Science Foundation Coursey D101. This is not to say

that the Course Team hopePtce lure students away from the Social

Sciences but rather it is hoped that more students *ill take both a

Social Scieece and a Technology Foundation Course. It was decided

therefOre to look at students who had been offered a plaoe on either

T100 Or D101 and who were waiting for their course to begin. Only new

students (i.e., those commencing their first year of studY as Open

University undergraduates) were regarded as relevant subjects for

the study. This was-because T101 will be oriented primarily.towards

the requiremeqts of students with no previous experience of university



Tabl.e.2. Percentage

31. 12. 77

Withdrawal of New Students froM T100 at

by Educational Qualifications.

Educational
Qualifications.

0

'.

Percentage of students
in this category who

, ..
have withdrawn from
the course

No formitl

-

45

I

CSE/RSA
1

et.

50

101 level 1-4
.

.

,

30

em level 5+ 28 ..

1A1 level 1 22

22

,

ONC irr OND

..1..

24

HNC or HND 16

Teachers Cert. 13

Univ. Diploma
6

15

A

Univ. Degree
. 4

No data
,

11

1 6

a



- 13 -

44

ldvel study. Consequently, it is necessary to pase the design of

1

the teaching imaterials on the skills and capabilities of liew

students rather than those already in the Open UniVersity kearning

system
a

a

It w40 decided not to survey the whole body of new students
. . . 1

registered for both

expected to possess

T100 and D101,sinc,d many of thohe can be

a
quite gophisticated,numeracy skills. The aim

of the study.lias to identiYy-the lowest likely levels of entry
t

skills and learning capabilities
w
of potential T101 students in order

a

to determine the starting leveland learAing gradient for such skills

inithe course. Applicalits for Open University courses are asked to

indicate Which of a number of educational qualiilcation categorieip

they fit into on entry. Table 2 lists these educational categories

ih the left-hand column.
:*

v

.

The Table-shows the4pfiriportion ot new students who hadstrithdrawn
.

from T100 in 1977 by the'end of the course-expressed as'a percentage.

of the total number of new students who had paid their course fees.

From this Tabld it can be seen that there is a tendency for students

with few educ,tional qualifications to withdraw.from T100 in greater-

proportions ihan those'with rather more qualifications.

a

It woaldecided to include students in the lowest three 'categories
,

% r

,

only in thelvamPle. The sample populaiion therefore was all 2279

iof the new tudents who had registered for either T100 o'i- D101 .in 1978

7
V

4



\I

and who fell into one of the followinft.three educational categories:

10

(1) Students with no formal educational qualifications.

(2) Students having a CSE, RSA or school leaving

certificgtte in one or more subjects.

(1) Students with GCE 20, level, CSE 202 grade, school

certificate or equivalent in 1-4 subjects.

4

Table I shows the proportions of students in each of these

categories as a percentage of the total intake of.newly registered

students to'both T100 and D101 in 1977. It is apparent from the

%Table that thp proportions were approximately the samefor both cases.

Taile 3. Proportions of Newly Rpgistered Student Intake 1977

4..

in Different Educational-Categories.

Educational COURSES

Qualifications . T100 D101

No formal 11% I0/4
.

.

csA/4A 2 4% ii%
,

-4

1-4 201 levels ,. ty% . to%

Total
,

28% 24%

8



0

Administration of the survey

It was decided that it would be impracticabl.e to ask any student

to attempt more than one questionnaire because of the amount of time

1'

requird to inswer each one. Pilot tests indicated that, for those

students who had to work through all. the deiaiked teaching and the

accompanying problems, each questionnaire required approximately'

2i hours study time to complete.. Accordinglyithe sample population

was split into six equally siied groups, each group receiving a

different quesiionnaire. Within each group tha three educational

categories sampled were represented in the same proportions as in,

the sample popula;tion as &whole. Each qmpstionnaire was accompanied

by a covering letter which explained the aims of the survey. It also

invited the recepient to return the questionnaire blank if ,they were

unable to complete it for sow, reasonl-but to specify what those

reasons were. No inducement.tarespond was offered other than

. an appeal to their- altruihni.
1.1

t

The first Iwo, questionnaires were mailed jto students in NoIrember t
#

1977 and the remainder were sent as soAres they were ready. The

last one was despatched at the beginning of rebruary 1978.

. THE SURVEYRESULTS

Responee rate.

At'

The response mite for Completed returned.questionnaires declined

gradually.from 66%,to 57%; with a mean response rate.of approximately

62%. This decline can probably be explAinedly the fact that

.foundation courses kegin in early February and in addition to the

actual course texts, stuaents receive a large volUme of material ,

from their regional office and other OU sources. consluentay, the



4

later queStionnaires would have been in increaciing compdtition with

other moeci pressing material for the htudents' Ittention.

Of the je% who did not return a completed ques:tionnaire, sows

responded to our invitation to return it blank with an explanation.

\

These responses were bllster.ed bycdditional coMillents volunteered

by .students who had returned completed questionnaires. The inter-

pretation and classification of such open-ended\reedback was to a

large extent arbitrary. Many respon*es contained' multiple reasons

for non-return. However the following four categories Amerged as

the major relevant ones:-

(1) Thoseo4ho regarded the.exercises as too daunting to attempt
*A- .

and those who attempted them anyway and wrote to us telling

us how difficult they were (28'responses).

(2) Those wtko for perSonal reasons such,as illness, changing job,

%moving home, etc. were unable to:find the time'to coM7iete,

the questiarnaire (21 responses).

(3) Those who did not understand what they had been asked-to

0

dol9 response).

(4) Those who regarditd the exercises.Set as too triyial to be

worthy of an attempt and those who attempted ihem anyway
I.

0 .

but *rote to us telling us how eapy they were (7 reaponses).

.

The misunderstanding* referred to in category 3 'above were

aimosi entirely due to the fact'that the D104students who received

a questionnaire assumed that we had sent it to them by mistake. Many

of these respondents told us th'at their difficulties with

mathematics had,contributed.to their decision never t6 take a

technojogy coUrse. This is interesting because such studnts are

precisely the kind we are tring to encourage to study T101.

20



* % of students getitng all the Prokess right

Fig. 2. Pei-formance bifferences Associated with Course Preference



performance differences associated with course preference

r 4 '

Differences between the T100 and D101 itudents are sDOwn in

Figure 2. It can be seen fromithe figure that on Average about 15%

more T106 students than 1)104 students got all therdblems right on

' their first try. In most cases this diffirence was reduc'ed to about

9% by the reminder and further reduced to about 6% by the detailed
.'

feaching. The two exceptions to this trend'were ihe Fractions and

Ratios questionnaire and the Percentages questionnaire. On the

.

Fractions and Ratios ques ionnaire, the iniV.ai difference between

T100 and D101 students wad-Tairly small (9%) and itliholied little

change after either the iteminder or the detailed iteaching. On the

Percentagits questionnaire, a larger initial differerice-Of 1%; was

eeduced to 16% by the reminder:and to 11% by the detailed teAching.

Performance differences associated with educational qualifications

if

IV

.
.

Performance ifferences associated with differences in educational
.

.

,
#

-qualifications are shown in Figure 3. On four of the sfx-questionnaires,

A larger proportion of respondents in the 14 101 level category got

all the answers right on'thetr-first tey than did eespondents in'

either.of the other two categoriesf on'the Graphk questioniiaire, .th9

- difference betwefin the 1-4 001 lievlitlo and the "no qualifications"
_

category was only 1%. On the two questionnaires where the peeformance
, .

.
is

-
, of the 1-4 00' levels group was not the best, Decimali and Areas lux!

.

- A -

Volumes, thedifferences_between all three categories were less than
. .

. e
.

_

396. guriiisingly,sponClents ufith no formal qualifications.did

slightlybe'fter than students in the; CSE/RSA category 9n the./first'

Ary in five of the six questionnaires.

There.was some tendency for'performance differences between

ir 22



Nallb

r

.

I
1

- .

% of students geiting all the problems right

4

30

26

4.

TRY 1' 2

No formal

6

e .

.1 2 3 1 2 ,3 TRX

AE/RSA 1-4 20/ levels

'BASIC ARITHMETIC

s'-

11

4.

$.

-

4.

a

A

moilt

Fig. 3: Performance Differences Associated with Educational
Qualificationi.

23



Joe.

4111r
of studisnts getting all the problems Aght

TRY . 1 - 2 3

No formal

I ,2 3 1 2 3 TRY

CSE/RSA 1-4 '0' levels

FRAdTIONS AND RATIOS

'Fig. 3 ,(Cont.)



ar -

cfa of students ng oil the probleasgetti

/-

Mo formai

Fig. 1. (Cont.)

1. 12 3

CSSALSA

VOINTAGES

1.-.6 *CIO levels "'

25

gt



44

1

.

% of stbdents gettihg all the'problens right

4

S.

qa

sr

WV 1 2 3 1 2 3 1 2, 1 Tur.

No form:al, esgittsAt 1-4 90, levels

Fig. 3. (Coot.)

DEC

26

a.

A



41.4

Mirs

4

% of students getting all the problems right

100-,

90-

0

80- ,

4d-A

30-;

20-1

10-k.
Q-

.

TRY 2 1 1. 2 ,47 ,
1 .

No 'formal -csAft.tsA .tot levels

'

4

2 3 TRY.

4 S ARRAS AND VOLUMES'.

filb: 3. (Cont.)

-r'""



. ."

% of students getting all the problems right

100-

70-

6o.

50-

*30-

20-

10-,

AV
re

A

Tiff 1' 2 3 -1 2 3 1 2 4 3 TRY '

csg/OsA 1-4 100 levelsNo. formal

GRAPHS

Fig...". (Cont.), .



_dr

0

categories to be reduced on successive tries, modt notably in'the

Basic Arithmetic questionnaire. It is important to note, however,

that in the Percentages questionnaire, the initial. 10% lead

established by the 1.14 109 levels yroup was barely diminished

despite the fact that the'other two groups converged. Similarly

in the Graphs questionnaire, the 1-4 ,levels and the no

qualifications groups began witgan approximately 6% lead over the

CSE/RSA group; a lead which they retained throughout successive tries.

Overall performance results

e,

Figure 4 summarises the overall

For eadh questiolmaire, it shows the

got all ofthe problems right on the

and following the detailed teaching.

performance results of the Study.

proportion of respondents who

first try, after the reminder,

'

It can be seen from Figure 4 that the reminders' were in all cases

effective, producing on average, a 20% increase in the proportion

of mispondents able to get all the problems right on their seopnd

try. The effects of the detailed teaching were less clear. Although

moderately effective on the Basic Arithmetic and Decimals questionnaires,

the overall increase in the proportion of students able to get all the

problems right after the detailedlteaching, was on average,:less than

5%. This left. some 10% of the respondents still unable to answer the

problems correctly, even after being exposed to the detailed vteaching.

Overall approximately 35% of the respondents did not get all the



e

%.of student6 getting all the rob1¼ns right

100-

90-

1 80-

60-(

30-

20-

1CM

11111M%

1 1

1. 1

T R Y 1 2 3 i 2 3 1 2 3 1 2 . 3 . 1 2 3 1 2 3 TRY

o miA ,gI; 1 4 d m
V 1o o

ci 2
r4 44 r4

I P.1

14 1 1 E VA 1,4 0-A g

Wm

Fig. 4. Performance on Bath Questionnaire on Successive Tries,

30



problems right on their first try. Assuming the.respondents to be

a
representative of the sample population, this suggests that 35%

of all new T100

categories sampl

(4From Table 3 itvcan be seen that students in the educational

V

D101 students in the three educational

need some help with the kinds of problems set.

categoriewsampled constitute 26% of the combined inUke of new

studitnts to T1.00 an4VE1101. Thus the proportion of new students

entering these courses who need help,with basic mathematical skills

ig! 35% of 26% Or just over 9% of the total intake of new students

into T100 and 1)104.

vascussIom OF THE RESULTS

In general the results of the study indicate that the algorithmic

skiils present difficulties for a significant proportion of students

in the categories sampled, particul.arly for those with few or no

academic qualifications and those Who have indicated a preference

for social science rather than techhology courses. The fact that

some (38%) of the students selected for the survey sample did not

yespoild obviously has iMplications for the representatiVeness of the

results obtained and for the validity of any conclusions drawn froth

those results. Although meagret, the open-ended feedback suggests

that a majir reason for non-response to the survey was a desire
by

individuals not to reveal their ladk of numeracy skills. The

feedbadk 'also suggests that students with numeracy problems

deliberately avoid the present Technology Foundati4n Coui.se precisely

*

because of this weakness. In other words., the perfotmance of those

students Who responded-is probably better on average thah the actual

mean performance capability of the Whole sample. In addition, it
'

11/

31

b.



seems 'reasonable to suppose that numeracy problems are not confined

to students'in the categories sampledrand that the problem is

eyen greater thim indicated by the survey results. For example,

students with-only one.otAt level or evdn a degree in an area not

requiring numeracy may have problems with mathematics. Similarly,

students who currentlg register for the Arts Fouridation Course may

do so.partly because they are reassured by its essentially non-

mathematical Syllabus.

The ineffectiveness of the detailed teaching used inillfte survey

is of particular concern to currivlum designers. As stated in the

introduction, the matbrial was drawn partly from a range /of Open

University sources and partly generated especially for the survey.

'One ()pious problem with the detailed teaching was the absence of

practise exercises. Several respondents remarked that they believed

they could have done better given more time to practice-the skills

beim; taught. Others actually told us that they had borrowed "Teach

Yourself Mathematics" types of books and set themselves practise

exercises,

Applying a 90% criterion-level to the results it has been'possible

to make firm recommendations to the Course Team abOut the level of

teaching which should be provided in T101 for each of the skills.

surveyed. These recommendations correspond to the three levels of

teaching provided in the questionnaires: none; a briei reminder;

detailed teaching. Thus if 90% or more of the respondents got all_

of the problems in a particular bloc& right on the first try then

the:recommendation for Rroviding no teaching was made. If less than

90% got all the problems in a block right on. the first try then the

32
#



reconsnendatid'in was for the provision of a brief reminder. Similarly

if less than 90% of thearespondents who tried the problems after the

reminder got them,all right then a recommendation was made to the

e Course Team for detailed teaching of the skill covered in that block.

In the light of the survey finding that on average 10% of the

respondents were still unible'to get all the problemexight after

the detailed teaching, four further iecommendations have been made:-

(1) Numeracy teething should be integrated into the. course material

in a meaningful way. Because the course is designed to be

issue and problem oriented, the need for calculations will

arise naturally from topics 1iecussed, cOncepts taught, and

evidence evalUated. It is anticipated that this integrtttion

will be more motivating, than for example, the kind of

preparatory mathematics booklets used on other Foundation Courses.

4

(2) Mathematical topics should be taught via the use of a calculator.

This will allow students to tackle problems Using realistic

data right from the start without having to go through the

tedious process of long computations. It is self-evident that

students will have to be taught how to .use the calculator in

a sensible manner.

(3) The first six weeks worth of.course work Which ceintains the

bulk of the basic numeracy teadhing should be mailed to students

early, so that instead of Six weeks new students will have the

option of spending two to three months practising and.consolidating

their basic numeracy skills.

(4) Optional practise exercises shonld be provided for the minority



of studenti who still have difficulty after the detailed

teaching but it should be stressed thatIthese are additional

to the course workload. That is to say, the Course Team shoul

not take the learning requirements of these students as its

baseline for developing numeracy skills but it should provide

extra optional helpjor the few who need it.

A number of important assumptions underlie the war in which the

Numeracy Study has been carried out. These concern the representativeness

- of the sample selected, the'extent to which the "material used in the

;

questionnaires is likely to correspond to the teaching material used

in the course, the validity of the comparisons drawn between different

groups within age sample and the reliability of the data obtained.

It has been assumed that the selected Sample is representative

ok the future student intaim into the new course. There is of course

an unavoidable conflict built into this approach. On the One hand,

it was our intention to establish what the skills and learning

capabilities of future T101 sttidents`will be immediatel7 rior to thefr
4

entry into the course. On the other hand, we wanted the information

sufficiently far in advance of the commencement of the course for it

to be useful in the preparation of the instructional material to be
.

used in the course. In practice this meant that the attidy had to be

darried out two years before the course is due to be presented to

the first students. Any attempt to base the design of new instructional

material on the known skills and abilities of-future students seems

likel to-encounter this problem unless the'candidates are carefully

selected for theif suitability to undertake the course. This latter,

approach is quite common, but it is arguable that students are then

being matdhed to the teadhing material - an approadh which is quite

34



antithetical to the spirit and aims pf the Open University.

The problems posed and the teaching material provided in the

questionnaires were each presented in a pure, abstract form divorced

from any practical application. But technolwists use mathematiCs

as a toot for tackling real problems. Consequently, ih T101

mathematics-wil be introduced and taught in the context of practical

applioationq. On the one hand it may make it easier for students to

learn theJbasic numeracy skills if they can see their relevance and

usefulness and apply them to concrete examples. Alternatively,

students may encounter-additionarNifficulties as a result of having

to learn how to interpret and remodel problems so that they can be

handled mathematically. ConseqUently, thefindings of the Numeracy

Study can,only be taken as a very rough'guide to the likely performance

capabilitiei ..d.hd learning difficulties of students actually studying

the course.

It has been assumed that the student gloups tackling the various

questionnaires were evenly matched in terms of their skills and fi

learning abilities and that comparisons between groups are thua valid.

It is impossible to be certain that the groups.are matched unless

individuals are assigned to them on the basis of their known skills

/
and abilities.

Sinc 1
these are what vie wish to measure and to do so

we havesto divide the students into groups first there is clearly a
.t.

conflict here. The best we can do is assume that within each of the

three educational categories identified, stpdents have approximately

eqUal capabilities and skills. They can then be assigned randomly to

groups in such a way as to ensure-that the relative proporitons of

students from each catedory are the same in all groups.



-Finalli, it has been assumed that the responses obtained are
A .

honest and accurate. It. is bonceivable that some students may have

cheated by looking up the answers and filling these in instead of

working thrliugh the problems set themselves. Some may ibelieve that,

despite our`assurance te the contrary, h high score would favourably

influence their future assessment record in the University.

Each of these four assumptions can be regarded as a weakness of

the_method used in the study in that they cast doubt on the

reliability of the data obtained and the validity of the conclusions

drawn there from. However, before passing judgement, the method has

to be reviewed in the context in which it was devised. The aim of

the study was to obtain only a first approximation of the likely

skins and learning capabilities of potential Tiol students. The

findings are intended to supplement the Course Team's teaching

experiences in an area beset by anecdote, hearsarand prejddice. As

iexplained in the Introduction, he Numeracy Study is one part of a

three part strategy for ensuring the coordination of students' needs

with course production. "he other two parts, developmental testing

and complete course evaluation, will provide the necessary checks

on the findings of.the Numeracy Study. The strength of the Numeracy

Study is that it is predictive. It provides information About

students' likely learning needs sufficiently far in advance'.to be

useful in the preparation of the course. The other two methods are

necessarily poet hoc4in their orientation. They can only provide

information about the suitability of material Which hes already been

epared.

In conclusion, we-believe that the findings of the Numeracy'StUdy



,

demonstrate the importance of carrying out e systematic evaluation

of Critical learner skills ad a pre-requisite to the design of

41,

instructional material. Without the knowledge generated by the

study, the T101 Course Team would have been in great danger of-
,

selecting an initial skill level and subsequent learning gradient

whith wouid be too geeat for at least 9% of the new students for

whom the course is being prepared. It is hoped that other

course designers at the Open University and elsehohere will be

encouraged to undertake simi].,fr types of investigations.'

11.0

ks1/

L

3 7

t, N



41.

COURSE DEVELOPME7.GROUP PAPERS,

CB1 NOTES ON THE EARLY. STAGES OF-COURSE PLANNING

by Phillipe Duchastels-Roger Harrison, Euan Henderson,

Barbara Hodgsont Adrian Kirkwood, Robert Zimmer.

CD2 HOW TO USE CONSULTANTS SUCCESSFULLY
by Judith Riley

CDI DISCUSSING AND EVALUATING DRAFTS'
by Judith Riley

V
CD4 ASSESSMENT

by Judith Riley

CD5 INTRODUCING NEW FACULTY MEMBERS TO COURSE PRODUCTION

by Phillipe Duchastel and Roger Harrison

CB() BRIDGING THE GAP BETWEEN TFACHING AND LEARNING AT A DISTANCE

by Michael Nathenson

C.1,7 CADI14gx1t4CT Ft-Ebehte,4 sway/ 6 °AMIE:
tape* rh Pt*

4,3 Via, 664.44toci

s -
P

38

11#

-/