LB UNIVERSITY OF PENNSYLVANIA IC-NRLF SB 17 ESD STANDARDIZED EXERCISES IN UNITED STATES HISTORY. COLONIAL PERIOD STURGISS BROWN DAVIS A THESIS PRESENTED TO THE FACULTY OF THE GRADUATE SCHOOL IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF PITTSBURGH PRESS 1921 EXCHANGE UNIVERSITY OF PENNSYLVANIA STANDARDIZED EXERCISES IN UNITED STATES HISTORY. COLONIAL PERIOD BY ii STURGISS BROWN DAVIS A THESIS PRESENTED TO THE FACULTY OF THE GRADUATE SCHOOL IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF PITTSBURGH PRESS 1921 TABLE OF CONTENTS 1. Introduction 5 2. Purpose of the Study 6 Chap. I. Selection and Organization of the Material to be Stand- ardized. Sec. 1. The Content of the Exercises 9 Sec. 2. The Form of the Exercises 11 Chap. II. Standardizing the Material. Sec. 1. Procedure Followed 15 Sec. 2. Schools Taking Test I 20 Sec. 3. Schools Taking Tests II and III 21 Sec. 4. Scoring the Exercises 28 Sec. 5. Tabulating the Results 29 Chap. III. Tentative Scales in the form of Lists X and Y 33 Chap. IV. Weighting the Results of the Data on each Exercise. Sec. 1. Plan of Procedure ." 43 Sec. 2. Tests for the Reliability of the Data 44 Sec. 3. The Reading Difficulty of the Exercises 50 Sec. 4. The Element of Success by Chance. Formula 52 Chap. V. Classification of the Standardized Material into Scales C and D and Tests E and F 55 Chap. VI. Use of the Standardized Exercises for Supervisory Purposes. Sec. 1. Reorganization of Proposed Measures into Test A and Test B, Part I and Part II 69 Sec. 2. Value of the Standardized Exercises 81 Sec. 3. Significance of the Value Found for Each Exercise. 85 Four Different Kinds of Historical Material 87 Sec. 4. Possible Uses of the Standardized Exercises 89 Chap. VII. Summary and Conclusion ' 95 Sec. 1. Summary 95 Sec. 2. Conclusion 96 Bibliography 97 62394 STANDARIZED EXERCISES IN HISTORY INTRODUCTION There are now available for the use of teachers a number of standardized tests in United States History. All of these tests cover in a general way the whole period of history and are especially use- ful for survey work. There is need, however, of a different kind of test for supervisory purposes which shall include the many details of a single period of history and at the same time lend itself readily for diagnostic purposes. Such tests have been devised and are here presented as one more means of determining scientifically the attain- ment of pupils in this subject. The plan used in standardizing the material of tests is given in more than usual detail in the hope that teachers of history and supervisory officers may be able to profit by the experience of the writer and extend the work here started. Care has been taken to exercise simplicity and directness at all times so that the student who is just beginning research work in education may find directly or indirectly answers to questions of statistical procedure. Briefly the study was carried out as follows : After determining tentatively the aims and limits of the investigation, the material to be standardized was selected from a source of unquestioned value. This material was then put in the form of exercises to be worked out by pupils. The responses of these pupils were then scored, tabu- lated, evaluated, and finally weighted in terms of the functions of the Normal Frequency Curve. These weightings confirmed the belief in the possibility of classifying the exercises into tests and scales. They also showed that there are four distinct types of historical material, characterized by the difficulty pupils have in comprehend- ing them. Such classifications were made both in form and content. While these classifications are correct theoretically it was deemed wise for practical purposes to consider only the approximate weight- ing of each exercise and to reclassify all the standardized material into two tests. In this form some accuracy was sacrificed but the reliability for supervisory purposes is just as valid. A Teacher's Manual was prepared which gives full directions for administering, scoring and using the results of the two tests formulated for school use. . - . > -, J . - - . .* - * .* 6 * Standardized 'Exercises in History The execution of this plan would not have been possible with- out the hearty co-operation of teachers of history, supervisors, and students of education. There was the most cordial response at all times and I wish here to express my appreciation of all the assist- ance I received however inconspicuous it may have seemed to the one giving it. Such general acknowledgment, however, is not just regarding the sympathetic guidance and suggestions I received from Dr. Harlan Updegraff of the University of Pennsylvania. He suggested the intensive study of the Colonial Period, and the distinctive source of the content of the exercises. He also gave me the benefit of many constructive criticisms before the manuscript was presented for publication. To my fellow students of the seminar group in Educational Administration at the University of Pennsylvania, I am indebted in more ways than I can express in a formal work of this kind. How- ever, it should be said that the constructive criticisms and pointed suggestions which they offered when the reports of progress were made from time to time on this study, served as a valuable check against biased thinking and as a stimulus to merit their approbation. PURPOSE OF THE STUDY The purpose of this study has been the derivation of stand- ardized testing material in United States History comparable in exactness with the contents of such material as the Monroe Reading Test, the Buckingham Spelling Scales, and the Ayres Handwriting Scales. The hypothesis was maintained that it is just as possible to measure certain aspects of content subjects, such as history, as it has been found to be possible to measure the attainments of pupils in the form subjects just mentioned. Limits of the Investigation For the purposes of this study and under the conditions it had to be pursued, it was deemed advisable to circumscribe the field of investigation, and to limit the method of work. Only in this way did it seem possible to secure reliable results in a field so little known. After a careful consideration of the possibilities of the situation the following limits were determined : Introduction 7 1. Informational Material. The content of the proposed tests should be confined to the informational aspect of the instruction in United States History. Full and due consideration was given to the other aims in teaching history, but the attainment of information was considered fundamental to the other aims. Dewey has shown clearly that information is one of the prerequisites to sound thinking on any problem. It seemed evident also that this aim could be measured more definitely than the other aims such as citizenship, open mindedness, method of thinking, etc. 2. One Period of United States History. The informational material used for the tests should be limited to a certain period of history, or a cross section of the whole course of study. It was thought that the material of a given period would be more homo- geneous and could be handled more accurately. The Colonial Period was chosen because it has always been taught rather thor- oughly in schools, and abounds in informational matter. There seems to be a need also for a series of tests each based upon a single period of United States History for the use of supervisors after a class has been given the usual school instruction. 3. A Regular Form. The information used for the testing material should be cast in a certain form in order that the scoring might be objective, and at the same time secure the probability of consistency and uniformity in preparing the tests, and the responses of pupils. Such uniformity would also be likely to economize time. 4. Eighth Grade Pupils. The tests should be administered only to the advanced division of pupils in the Eighth Grade. The purpose in confining the data to this class of pupils was to find out as nearly as possible the net results of school instruction relating to historical information. The belief was maintained that whatever information pupils had, at least one year or more after completing the course in the Colonial Period, represented material which is easily comprehended and retained ; conversely, that material which is difficult to learn would either be wanting or so vague in the minds of the pupils as to be lacking for ready command. CHAPTER I THE SELECTION AND ORGANIZATION OF THE MATERIAL TO BE STANDARDIZED After the purposes and limits of a study of this kind have been determined the next points to be considered are the source of mate- rial and the form in which the content is to be placed. Unless this is done the investigator must be prepared to expect all sorts of irregularities in the responses of pupils. SEC. 1. Content of the Exercises. The reliability of the re- sponses of children as data for standardizing material of instruction is in proportion to the opportunity children have had to learn the content of the test. The difficulty of finding such material for this investigation was solved by the use of the very careful study by Bagley and Rugg. This study appeared as Bulletin No. 16 of the School of Education of the University of Illinois. The authors took twenty-three American Histories then in common use and made a statistical study of the amount of space devoted to the different topics, "common to at least seventy-five per cent, of the books" and additional topics "common to at least fifty per cent, of the books." Tables V and VIII of Bulletin No. 16 include topics devoted to the "Period of Colonial Settlement and Development" and to the "Colonial Wars." The topics in these tables form the source of the content of the exercises used in this study. As the study progressed it was evident that additional exercises could be devised which were entirely within the range of pupils' interests, and as fair to them as the exercises based upon topics taken directly from the Bagley and Rugg study. The answers to these exercises were all taught in school in some form though seldom if ever asked for directly. The ability to work them out was more of an application of common sense in using the facts already acquired. The following is a typical illustration: "By the time the Mayflower had arrived at Plymouth Rock it had crossed the Atlantic, Antarctic, Arctic, Indian, Pacific ocean." "By the opening of the Revolutionary War the number of colonies had increased to twelve, thirteen, fourteen, fifteen, sixteen." Since these exercises were worked out successfully by a large percent- age of the pupils tested, and no objection was raised by teachers, the 10 Standardized Exercises in History author believed he was justified in including them and others of the same type. (Copy of Test I is shown beginning on page 18.) To aid the reader in gaining a more adequate conception of the wide range of information required to work out intelligently the exer- cises of Test I the following classification of the items of informa- tion is here submitted: NOTED CHARACTERS 1. Andros 15. Davenport 29. Mather 2. Argall 16. Delaware 30. Oglethorpe 3. Baltimore 17. Dixon 31. Penn 4. Berkley 18. Edwards 32. Pocahontas 5. Berkeley 19. Endicott 33. Powhatan 6. Bacon 20. Gorges 34. Rolfe 7. Bradford 21. Hooker 35. Say 8. Brewster 22. Hutchinson 36. Smith 9. Calvert 23. Kieft 37. Standish 10. Canonicus 24. Leisler 38. Stuyvesant 11. Carteret 25. Locke 39. Williams 12. Charles 26. Mason 40. Winthrop 13. Claiborne 27. Massasoit 41. Yeardley 14. Dale 28. Minuit RELIGIOUS SECTS 1. Baptists 7. Moravians 2. Catholics 8. Pilgrims 3. Episcopal 9. Puritans 4. Dutch Reformed 10. Quakers 5. Huguenots 11. Congregationalists 6. Methodists COLLEGES 1. Brown 4. Princeton 2. Harvard 5. William and Mary 3. Penn 6. Yale EARLY LARGE CITIES 1. Baltimore 4. Philadelphia 2. Boston 5. New York 3. Charleston NATIONALITIES 1. Dutch 5. Irish 2. English 6. Scotch 3. Germans 7. Spanish 4. Indians 8. Swedes 1. 1607 2. 1619 3. 1620 4. 1643 5. 1683 Selection and Organization DATES 6. 1733 7. 1754 8. 1775 9. 1787 10. 1789 11 1. Bacon's Rebellion 2. Charter Oak 3. Charter Colonies 4. Claiborne's Rebellion Colonial Wars 5. Anne 6. William 7. George 8. French and Indian 9. Democracy 10. Monarchy 1. Connecticut 2. Delaware 3. Georgia 4. Maryland 5. Massachusetts 6. New Hampshire 7. North Carolina 1. Boston News Letter 2. City of Brotherly Love 3. Debtors of England 4. First Colony 5. Indigo 6. Indentured Servants 7. Holy Experiment 8. Last Colony 9. Log House 10. Mayflower 11. New England LEGAL TOPICS 11. Aristocracy 12. Autocracy 13. Oligarchy 14. English Crown 15. Continental Congress 16. Grand Model 17. Great Treaty 18. Mason and Dixon Line 19. Royal Government 20. Toleration Act EARLY COLONIES 8. New Jersey 9. New York 10. Pennsylvania 11. Rhode Island 12. South Carolina 13. Virginia 14. Maine MISCELLANEOUS 12. Patroons 13. Plymouth Rock 14. Poor Richard's Almanac 15. Preachers' Influence 16. Rice 17. Slaves 18. Tobacco 19. Witchcraft 20. Jamestown 21. Corn Total 136 SEC. 2. Form of the Exercises. After the content of the test- ing material had been decided upon it was necessary to consider the form in which it should be put. The responses of pupils had to be evaluated in an objective manner. 12 Standardized Exercises in History The studies of Starch, Monroe, Kelly and others all show how variable are the marks of teachers if there is opportunity for a differ- ence of opinion. The ordinary "question and answer" method of testing pupils was at once discarded for the usual difficulties would be inevitable. Either doubtful or partial answers would have to- be accepted or the content be limited to those few points about which teachers and pupils readily agree there can be but one acceptable answer, such as dates. The "completion blank" form was considered and tried in a preliminary test. It was not found satisfactory, for either the pos- sible acceptable answers had to be listed arbitrarily or a shifting standard constantly defended. It was noted also that pupils made wild guesses, and that it would be very difficult to frame up state- ments which did not admit of two or more interpretations. One has but to try this plan to see how ingenious pupils are in inventing answers which represent a partial truth. At least they contain enough excellence for pupils to stand up and defend their answers. A cancellation test suggested from those used in psychological work was tried, but rejected because of the unnecessary length of time required to indicate the correct answer. Finally it was decided to put the desired response in each case along with seemingly correct responses and ask the pupils to indicate in some way their choice. At first they were asked to underscore the correct answer, but later it was found that it facilitated the scoring to have them draw a line clear around the correct one of the several suggested answers. A trial with this form proved satis- factory from the first. Pupils readily saw what was wanted, teach- ers approved it, and the grading was easily and objectively done. As the work of forming the material into tests progressed it became evident that standards could be set up which if observed would greatly facilitate subsequent efforts in this line. The follow- ing were derived: 1. Instead of asking questions, make statements called "exercises" in History. 2. The form and content of these exercises should be unmistakably clear and as easy to comprehend as possible. Selection and Organization 13 3. The vocabulary employed in the exercises should be such as is used in standard text books for elementary schools. 4. Sufficient suggested answers should be given to minimize the effect of chance selection. 5. An alphabetical arrangement of the suggested answers should be observed to avoid any constant error in placing of the correct answer. 6. Pupils should indicate the correct answer by drawing a line clear around it. 7. Any abbreviations used should be those in common use. 8. The answers offered should conform to the period under con- sideration. 9. The suggested ansxvers should be made up of teaching mate- rial, and any one of them seem a plausible answer to one not informed on the subject. Absurd suggested answers defeat the plan. 10. The suggested answers should be so different from the correct one that any quibbling over partial truths is precluded. 11. It should be clearly understood that one answer and only one will be accepted as correct. 12. No exercise should be submitted unless there is reasonable evi- dence that pupils have had the opportunity to learn the facts needed. The last five of these standards refer more specifically to con- tent but were here included for convenience since both form and content must be considered together in making up testing material. The term, "Exercises in History," was adopted because it seemed to be the most fitting one to use. The material was not put in the usual statement form and it was not in the form of questions. Pupils were accustomed to the word exercise as used in Arithmetic, Geog- raphy and Language. In each of these instances there was an answer to find. The situation is similar in the Exercises in History. A num- ber of answers are given and the pupil is expected to find the correct one. CHAPTER II STANDARDIZING THE MATERIAL SEC. 1. Procedure Followed. Haying determined the content and the form of the testing material it was next necessary to deter- mine the specific and relative difficulty of each one of the exercises. Two plans have been used by research students in standardizing material. One plan is to obtain the combined judgment of teach- ers whose experience and training seemingly qualifies them to render expert opinion as to the value of given questions, or the quality of work done by pupils. The Harvard-Newton Composition Scale and the Thorndike Handwriting Scale are outstanding examples of this plan as to quality of work. The other plan is to submit the mate- rial to pupils and grade the work according to standards acceptable to all. The assumption is made that, that material is easy which is easily, readily and correctly worked out; conversely, that testing material is difficult which is not easily, readily and correctly worked out by the pupils. The second one of these plans \vas adopted for standardizing the historical material used in this study. Each of the exercises was then submitted to pupils of the eighth grade in well recognized public schools. Since the number of responses which were supposedly needed, was greater than the writer could procure from pupils directly, a form letter and directions for administering the test were prepared and sent out to the chief school official of the system from which data were desired. A copy of Test I was always included with the letter so the administrative officer could see just the nature of the data expected. There was a hearty response for co-operation in every instance. In so far as it was possible the writer adminis- tered the tests himself. The form letter and directions for "Classroom Procedure" are as follows: EXERCISES IN UNITED STATES HISTORY COLONIAL PERIOD To Teachers of U. S. History: You are in a position to co-operate in an experimental study of the historical matter now generally taught in our Elementary 15 16 Standardized Exercises in History Schools. The Exercises herewith enclosed is the result of a prelimi- nary study which, it is hoped, will result in definite suggestions for class room work. Will you kindly assist in securing actual results from the pupils in your charge? The appended Classroom Procedure for administering the Exercises, and the Directions to Pupils on the first page of the fold- ers, are intended to make clear all necessary points. Please read these over carefully, and follow them as given. The material readily lends itself to the game idea and it is sug- gested that it be so used. A correctly marked copy is enclosed for your own use. Please do not mark any of the pupils' work, however. In working up the results of this study each school will be designated by number only. No personalities whatsoever will enter into the summary. The median score for all schools listed, and the median for your school will be sent you. Your assistance will be greatly appreciated by those responsible for the results of this study. Return postage is enclosed to cover cost of all material sent out. Address School of Education, University of Pennsylvania, Care of Dr. Harlan Updegraff, Box 11, College Hall. CLASSROOM PROCEDURE 1. All pupils who are going to participate should clear their desks, and provide themselves with pencils. 2. READ TO PUPILS: "Today we are going to have a new way of finding out how many facts of the Colonial History of our country we know really well. In order that all may have the same opportunity of making a good score, no one should start the game until I give the signal. I shall now pass out the papers with the first page upwards. You may look at this but nothing more until I say, read y." 3. Distribute the Exercises. 4. Direct the pupils to fill out the blanks at the top on the first page. 5. Read over with the pupils the illustrative exercises. If neces- Standardizing the Material 17 sary put one or more of them on the black-board and work them out so as to make sure pupils know exactly just what is to be done. 6. When all is in readiness for a period of uninterrupted work, say, Ready, GO! 7. Note down the time when the class begins working. 8. As the pupils hand in their papers put down the number of minutes each worked on the blank space for that purpose. Test I UNIVERSITY OF PENNSYLVANIA SCHOOL OF EDUCATION EXERCISES IN UNITED STATES HISTORY COLONIAL PERIOD Directions to Pupils: Fill in these blank lines with: Your name School Grade City Date Are you a boy or girl ? 1. On the following pages are a number of exercises similar to these. (a) Columbus discovered Africa, America, Asia, Australia. (b) The American Revolution began in the year 1763, 1765, 1775, 1776. (c) The first American colony founded by the English was Conn., Ga., S. C., Va. If you draw a line around the correct one of the several suggested an- swers in italicized letters for each Exercise, historical facts may be read as follows: (a) Columbus discovered America. (b) The American Revolution began in the year 1775. (c) The first American colony founded by the English was Va. There is but one correct answer in each Exercise. Try to find the correct answer in each Exercise whether you are sure of it or not. Hand in your paper as soon as you have worked out all the exercises you can. Take as much time as you need, but no more. Number of minutes you worked (Your teacher will note this.) 18 Standardized Exercises in History HISTORY OF THE UNITED STATES COLONIAL PERIOD Test I EXERCISES 1. The Mayflower was a chapel, hall, hotel, plant, queen, ship. 2. Miles Standish led in fighting the Dutch, Indians, Puritans, Swedes. 3. Roger Williams founded the colony of Ga., Md., Mass., N. C., N. /. Pa., R. I., S. C., Va. 4. The Patroons were Dutch fishermen, fur traders, landowners, miners, preachers, teachers. 5. Thomas Hooker led emigrants from Mass, to found Conn., Del., Ga. r Md., N. J., N. Y. 6. The first college founded was Brown, Harvard, Pennsylvania, Princeton, William and Mary, 'Yale. 7. The Witchcraft delusion occurred among the Baptists, Catholics, Dutch, Indians, Puritans. 8. John Berkley and George Cartaret once owned Conn., Del., Ga., Md., Mass., N. H., N. J., Pa. 9. The principal native food crop was barley, corn, oats, potatoes, rice, rye, wheat. 10. The Mason and Dixon Line was established between Pa. and Del., Md., N. J., N. Y., Ohio. 11. Nathaniel Bacon led a Rebellion against the tyranny of Governor A r tjall, Andros, Berkley, Dale, Minuit, Winthrop, Yeardley. 12. The Charter Oak was a church, meeting house, painting, school, town hall, tree. 13. The Grand Model was drawn up by Cecil Calvert, King Charles, John Locke, John Smith. 14. Of the nationalities represented in all the colonies the most numerous were the Dutch, English, French, Germans, Irish, Scotch, Swedes. 15. The Proprietary form of government was exercised in Mass., N. Y., Pa., R. I., S. C., Va. 16. The most influential professional class in New England were the editors, lawyers, physicians, preachers, teachers, writers. 17. The Toleration Act in Md. was decreed by the Episcopalians, Catholics, Purtians, Quakers. 18. To help the poor debtors of England James Oglethorpe founded the colony of Conn., Del., Ga., Md., N. J., S. C., Va. 19. The Pilgrims were kindly received by Chief Canonicus, Massasoit, Philip, Powhatan. 20. The first settlement by the Swedes was in Conn., Del., Ga., Md., Mass N. C., N. H., N. Y. Standardizing the Material 19 21. Slavery was first introduced in 1607, 1619, 1620, 1643, 1683, 1775, 1787. 22. New York was founded by the Dutch, English, Frencli, Spanish, Swedes. 23. The main export crop of early Va. was corn, cotton, indigo, rice, tobacco, wheat. 24. The Great Treaty was between the Indians and Governor Berkley, John Endicott, Cotton Mather, William Penn, John Smith, Miles Standish. 25. Pocahontas is said to have saved the life of Sir William Berkley, Jona- than Edwards, William Penn, John Rolfe, John Smith. 26. The Charter form of government was enjoyed by the people of Del., Ga., Md., Pa., R. I., Va. 27. The Huguenots first settled in Baltimore, Boston, Charleston, New York, Philadelphia. 28. The "Plymouth Rock" is in Conn., Del., Md., Mass., N. J., N. Y., Pa., R. I., S. C. 29. The first Pilgrims landed in 1607, 1619, 1620, 1643, 16S3, 1776, 1789. 30. William Claiborne led a Rebellion against Lord Berkley, Lord Brook, Lord Baltimore, Lord Delaware, Lord Say. 31. Indentured servants and slaves were held mostly in Del., Ga., Mass., Pa., S. C., Va. 32. As Royal Governor of New England Sir Edmond Andros represented the Continental Congress, the English Crown, the Proprietors, the Common People. 33. The First Representative Assembly was held in America in 1607, 1619, 1620, 1643, 1754. 34. William Bradford was once Leader of the Baptists, Catholics, Pilgrims, Quakers. !5. The first settlement of the Catholics was in Conn., Del., Md., Mass., N. C., N. H., N. J., R. I. 36. Jamestown was founded in 1607, 1619, 1620, 1643, 1733, 1754. 37. The first Newspaper in America was the Boston News Letter, New York Times, Philadelphia Ledger, Poor Richard's Almanac, Providence Journal. 38. The houses of the first colonists were generally built of brick, cement, dirt, marble, logs. The last of the thirteen colonies founded was Conn., Del., Ga., Md., Mass., N. C., N. Y., Pa., S. C., R. I., Va. In emigrating the colonists generally went east, north, south, west. The last of the Dutch governors was Hudson, Minuit, Stuyvesant, Win- throp, Yeardley. The Germans settled mostly in Md., N. J., N. Y., Pa., R. I., S. C., Va. 20 Standardized Exercises in History 43. The City of Brotherly Love was founded by William Berkley, George Calvcrt, John Endicott, William Penn, John Smith, Miles Standish. 44. The most important of the Four Inter-Colonial Wars to the colonists was Queen Anne's, King George'Sj French and Indian, King William's. 45. By the middle of the 18th century the form of government most de>ired by the colonists was aristocracy, autocracy, democracy, monarchy, oligarchy, 46. The Quakers in Mass, were persecuted by the Baptists, Methodists, Moravians, Puritans, Dutch Reformed. 47. The Colony of Ga. was once invaded by the Dutch, French, Irish, Spanish, Swedes. 48. The colony of Me. was once under the proprietorship of John Davenport, Sir Ferdinando Gorges, Anne tlutchinson, William Kieft, John Davenport. 49. The "Holy Experiment" was tried out by William Bradford, William Brevjster, Jacob Leisler, Peter Minuit, William Penn. 50. When the colonists first landed in America they found the natives to be Caucasians, Chinese, Indians, Malays, Negroes. SEC. 2. Schools Taking Test I. In the spring of 1918 the writer sent out or administered personally approximately 1,500 copies of Test I to schools in Massachusetts, New Jersey, Pennsylvania, Ohio, Indiana, Illinois, and Minnesota. Those administered by the writer were in schools in or near Philadelphia. The returns from a distance came from school officials who were interested in educational progress and there is every reason to believe that the test was admin- istered according to directions. In all 1,250 papers were returned r corrected and tabulated by the writer. The others were lost in one way or another. In the spring of 1919 the writer administered personally approximately 1,500 more of the same test in schools in and near Pittsburgh. This district is distinctly cosmopolitan, and has the further advantage for experimentation of variation resulting from the complex school organization of former years. Allegheny County is so populous and wealthy that many independent school districts are supported. The "State of Allegheny" has been aptly applied to this region. Even in the city itself, Pittsburgh, independent char- acteristics of instruction may yet be noted among those holding over from the ward system, although that form of organization was years ago supplanted by a modern unified system. The results from Standardizing the Material 21 this region within a few miles distance show as wide a variability as those gained from different states. When the data from this comparatively limited area had been scored, tabulated and weighted, they were found to be as valuable as the data coming the previous year from the seven states extending as far east as Massachusetts and as far west as Minnesota. This iie\v material was all the more striking since it was procured in different years. Table I shows how closely the results for the two different years and from such widely separated school systems correspond. Using the Pearson cos. method of correlation in which r=cos __ be the coefficient is found to be 1.00. If Sheppard's method of unlike signs be applied in which r=cos . H , the coefficient is found to be 1.00. Of course these formulae are for approximate results, but they are convenient for giving a quantitative statement about the data in this table which by inspection shows remarkable agreement. By Spearman's 'Toot Rule," r~ .98. The data producing these results came from pupils who were ready for high school in the following June. The values in Table I for 1918 are based upon the errors made by 1,250 pupils; the values for 1919 are based upon the errors made by 1,250 additional pupils, 2,500 in all. Table II is a modification of Table I and brings out other inter- esting details explained in the table headings. Fig. I shows in a graphic form the distribution of errors made by the 2,500 pupils. The close approximation to the Normal Curve of Error may be noted. Taking the test as a whole the addition of the data in the second year made but few striking changes. In many cases the rank- ing is identical. The algebraic sum of the shifting differences is zero. The author had additional data from pupils to add to that gathered in 1919, but when the results turned out so little different from those of the previous year it was not considered necessary to use them. SEC. 3. Schools Taking Test II and HI. Although the re- 22 Standardized Exercises in History TABLE I COMPARISON OF THE SIGMA VALUES FOUND FOR EACH OF THE EXERCISES OF TEST I No. of Exercise Value 1918 Value 1919 No. of Exercise Value 1918 Va'ue 1919 1 .53 .43 26 3.00 3.03 2 1.94 2.05 27 2.84 2.77 3 1.96 2.16 28 1.90 1.95 4 2.43 2.44 29 2.25 2.37 5 2.31 2.39 30 3.15 2.7'- 6 2.40 2.37 31 2.36 2.31 7 1.98 2.12 32 2.42 2.38 8 3.39 3.32 33 3.43 3.36 9 1.75 1.85 34 2.76 2. SO 10 2.48 2.46 35 2.28 2.34 11 2.39 2.39 36 1.85 2. OS 12 1.50 1.46 37 3.49 3.32 13 3.49 3.35 38 .7 : . .73 14 1.57 1.64 39 2.62 2.66 15 3.03 3.01 40 2.05 1.94 16 2.65 2.61 41 2.13 2.04 17 2.86 2.77 42 2.72 2.51 18 2.25 2.29 43 1.46 1.41 19 1.99 2.04 44 1.59 1.58 20 2.55 2.55 45 1.50 1.40 21 2.03 2.26 46 2.23 2.28 22 1.27 1.35 47 2.42 23 1.73 1.86 48 3.26 24 1.43 1.41 49 2.24 25 1.19 1.24 50 .51 suits from Test I were satisfactory the exercises were too uniform in value to construct a series of standardized tests or scales. By reference to Table II it can be seen that the nearest approach to the zero value is .43 sigma; and the nearest approach to the other ex- treme of the scale, 5.0, is 3.36 sigma. In the fall of 1919 Test II was devised with the purpose of rinding, if possible, well recognized material whose value would be more varied than that yet discovered. Some exercises were then devised which were thought to be very easy for pupils to work out, and others which were supposedly very difficult. The study of Bagley and Rugg was the source of the material as before, and it was sanctioned by teachers as being fair for the pupils. The same form was used as previously except that the number of suggested answers was kept uniform. Eleven different schools were chosen for Standardizing the Material 23 TABLE II COMPARISON OF VALUES FOR THE EXERCISES OF TEST I FOR TWO YEARS Number of Each Exercise Sigma Value in 1919 Sipma Value in 1918 Rank in 1919 Rank in 1918 Difference in Ranking for the two years D4- D- 1 .43 .52 1 1 38 .73 .75 2 2 25 1.24 1.19 3 3 22 1.35 1.27 4 4 45 1.40 1.50 5 7 2 24 .41 1.43 6 5 1 43 .41 1.46 7 6 ' 1 12 .46 1.50 8 8 44 .58 1.59 9 10 1 14 .64 1.57 10 9 1 9 .85 1.75 11 11 23 .86 1.78 12 12 40 .94 2.05 13 20 7 28 .95 1.90 14 14 41 2.04 2.13 15 21 6 19 2.04 .99 16 18 2 2 2.05 .94 17 15 2 36 2.05 .85 18 13 5 7 2.12 .98 19 17 2 3 2.16 .96 20 16 4 21 2.26 2.03 21 19 2 46 2.28 2.23 22 22 18 2.29 2.25 23 24 1 31 2.31 2.36 24 27 3 35 2.34 2.28 25 25 29 2.37 2.25 26 23 3 6 2.37 2.40 TT -/ 29 2 5 2.38 2.31 28 26 2 32 2.38 2.42 29 30 1 11 2.39 2.39 30 28 2 4 2.44 2.43 31 31 10 2.46 2.48 32 32 42 2.51 2.72 33 36 3 20 2.55 2.55 34 33 1 16 2.61 2.65 35 35 39 2.66 2.62 36 34 2 30 2.74 3.15 37 42 5 17 2.77 2.86 38 39 1 27 2.77 2.84 39 38 1 34 2.80 2.76 40 37 3 15 3.01 3.03 41 41 26 3.03 3.00 42 40 2 8 3.32 3.39 43 43 37 3.32 3.49 44 46 2 13 3.35 3.49 45 45 33 3.36 3.43 46 44 2 Total 36 36 24 Standardized Exercises in History PERCENT OF ALL PUPILS Standardizing the Material 25 the experiment which were in every way comparable with the schools in which Test I was given. The same directions were followed as before, and the work was all done by the author. Especial care was taken in administering, scoring and tabulating the results in order to make the outcome as accurate as possible. In spite of all the possible precautions the net results of this work revealed no exercises to be of lower value or decidedly higher value than were found in Test I. It was therefore necessary to devise another test which was des- ignated as Test III. The same methods of working were used as before except that the pupils were permitted to grade their own papers first, and all the material needed for weighting the exercises was gathered in the school room according to a plan to be given in detail in Chapter III. The results were more satisfactory. All the exercises in Tests II and III were then carefully evaluated and considered as preliminary values to forming two tentative scales. In all 1,102 pupils had been tested in 18 different schools. At this point are given copies of Tests II and III. The same directions were followed in each test. Test It EXERCISES IN COLONIAL HISTORY Draw a line around the one italicized word which is needed to turn each exercise into a historical fact. 1. The "Mayflower Compact" was drawn up by the Baptists, 1 Huguenots, Methodists, Pilgrims, Quakers. 2. When the earliest colonists arrived in America they were wel- 2 corned by the Caucasians, Chinese, Indians, Malays, Negroes. 3. George Washington was fitted for leadership in the Revolutionary 3 War through his experiences in the Colonial War of Queen Anne, King George, French and Indian, King William. 4. By the opening of the Revolutionary War the number of colonies 4 had increased to twelve, thirteen, fourteen, fifteen, sixteen. 5. The first attempt of the English to colonize in America was in what 5 was later called the colony of Conn., Del., Mass., Penna., Va. 6. The territorial claims of the English in America were established 6 by the treaty of Aix-la-Chapelle, American, Paris, Ryswick, Utrecht. 7. The period of colonization took place largely in the period begin- 7 ning with 1400, 1500, 1600, 1700, 1800. 26 Standardized Exercises in History 8. The political control of the Dutch in America came to an end with 8 the surrender of Governor Stuyvesant to the English, French, Germans, Spaniards, Swedes. 9. "Braddock's Defeat" occurred in the first, second, third, fourth, 9 Colonial War. 10. The first written constitution springing from the people and creat- 10 ing a government was in Conn., Del., Mass., Penna., Va. 11. The French army built menacing forts in Western Conn., Mass., 11 Md., N. J., Penna. 12. The leadership of John Winthrop was noted in the founding of 12 Baltimore, Boston, Charleston, Philadelphia, Providence. 13. The most outspoken friend of the colonists in the English Parlia- 13 ment was Dinwiddie, Montcalm, Pitt, Townshend, Wolfe. 14. Lord Delaware lived in England nearly all the time he was gover- 11 nor of Delaware, Massachusetts, New Jersey, Rhode Island, Virginia. 15. The "Pilgrim Fathers" came from England, France, Germany, 15 Spain, Sweden. EXERCISES IN UNITED STATES HISTORY Test 111 COLONIAL PERIOD Draw a line around the one italicized word which is needed to turn each exercise into a historical fact. 1. The Pilgrim Fathers came from England, France, Germany, Spain, 1 Sweden. 2. The last colony was founded in the 15th, 16th, 17th, 18th, 2 19th, century. 3. The New Sweden of 1638 later became known as the colony of 3 Conn., Del, N. H., Pa., R. I. 4. Among the many hardships of the time were the unexpected attacks 4 by Arabian, Chinese, Cossack, Indian, Negro warriors. 5. More lasting and prominent political events took place in Baltimore, S Boston, Charleston, New York, Philadelphia than in any other city. 6. The colonists were assisted in the French and Indian Wars by the 6 Iroquois, Mohegan, Narragansett, Pequot, Wampanoag Indian tribe. 7. The right of woman suffrage was granted for a time in Conn., 7 Del., Mass., N. J., Va. 8. Local business affairs in the New England settlements were gen- 8 Standardizing the Material 27 erally managed by the governor, king, mayor, minister, town- meeting. 9. The welfare of the New York colony was much helped by the very 9 early settlement of the Huguenots at Albany, New Burg, New Rochelle, Saratoga, West Point. 10. The strict Puritan church of early New England gradually gave way 10 to what was later known as Baptist, Congregationalist, Lutheran, Methodist, Spiritualist Church. 11. The light at night in these early homes came from candles, elec- 11 tricity, gas, kerosene, torches. 12. The Quakers founded Ga., Mass., N. H., Pa., S. C. 12 13. By the time the Mayflower arrived at Plymouth Rock, it had crossed 13 the Antarctic, Arctic, Atlantic, Indian, Pacific ocean. 14. The most common means of travel by the earliest settlers was by 14 aeroplane, railroad, steamboat, trolley lines, walking. 15. The United Colonies of New England was formed in 1619, 1630, 15 1643, 1683, 1763. 16. All the colonies were founded by men from Africa, Asia, Aus- 16 tralia, Europe, South America. 17. The "Thirteen Original Colonies" were once all under the political 17 control of England, France, Holland, Spain, Sweden. 18. Pocahontas was an English, French, German, Indian, Spanish girl. 18 19. Lord Delaware lived in England nearly all the time he was gov- 19 ernor of Del., Mass., N. J., R. L, Va. 20. The first written constitution springing from the people and creating 20 a government was in Conn., Del., Mass., N. J., Va. When you have finished turn your paper over. Number of errors It will be noted that there is some overlapping in the two tests. This was done to see if certain exercises would retain their rank when conditions were kept as uniform as possible ; and further to see if a few hundred pupils are all that is necessary to ascertain the relative difficulty, or the ease of a given point in history. That the exercise, "The first written constitution springing from the people and creating a government in America was in Conn., Del., Mass., Penna., Va." had a value of 3.55 in the first 11 schools and a value of 3.49 in the succeeding 7 schools was not due to mere chance. The experience with this exercise and the others used in both tests showed that if the material is carefully chosen, administered scien- 28 Standardized Exercises in History tifically, and a proportional sampling carefully guarded, only a few hundred pupils are necessary to tell the relative difficulty of a list of historical points. Attention is called to the dual numbering of the exercises in Tests II and III and Lists X and Y. The numbering at the right was used by the writer in checking over the corrections made by pupils. The point to this device is that the writer found it more convenient to indicate his marks on the right hand side of the page where the right hand never obstructs the view in a quick survey of the page. This last suggestion may seem like a trivial point but the observance of such details facilitates the work when the scoring runs up into the thousands of exercises to correct. SEC. 4. Scoring the Exercises. The scoring was done by the writer or by competent assistants under his direction. In all there were about 125,000 responses of pupils to check over respecting Test I alone; 6,000 for Test II; 14,000 for Test III; and 46,000 for Lists X and Y. Counting the control checks that were used respect- ing high school and normal school students more than 192,000 pupils' responses were taken into consideration in the scoring to produce the results desired. In the case of Tests I and II each exercise was looked over to see if it had been worked correctly or not. If an error had been made a straight line was drawn obliquely downward at the left of the number of the exercise in the case of Test I ; but to the right in the case of Test II since the numbers of the exercises were written on both sides. This kind of mark was used because it is quickly drawn and easily discerned, being away from the hori- zontal of the printed lines. Only errors were counted. They were less in number taken on the whole ; only errors were considered in weighting ; and the errors are the points of consideration in diagnosis. The number of errors on each paper was totaled and the sum placed at the lower right hand corner of the front page. As the work of scoring so many thousands of responses soon borders on the line of drudgery, several methods were used to expe- dite the work. Stencils were used such as are now used in scoring some of the current Intelligence Tests; a certain rhythm w r as main- tained by the use of a phonograph; but the most effectual plan Standardizing the Material 29 devised was to commit to the point of mechanical memory the correct answers as they should appear in order in running down the exami- nation paper. Any departure from the regular order or correct responses was thus readily and immediately detected. In Lists X and Y where pupils corrected their own papers it was necessary only to take samplings at random and unusual scores to see if the correc- tions needed to be checked over. In Chapter III is given the detailed directions which were used when pupils were permitted to mark and check over their own work. SEC. 5. Tabulating the Results. After the papers had been scored it was necessary to tabulate the results so they could be analyzed and interpreted. In this part of the statistical procedure the writer aimed to secure accuracy, speed, simplicity and the possi- bility of checks at as many points as possible. The plan adopted for this study is illustrated in Fig. 2. The tabulations are the actual results taken from the errors made by 22 pupils to whom Test II was given. PUPILS' PAPERS BY NUMBER r * ! i i ! l / / / 7 '7 / / !/ / / 7 7 Vf / JL fr*t*{ FIGURE 2 SHOWING FORM IN WHICH DATA WERE TABULATED FROM TEST I 30 Standardized Exercises in History These tabulations of school by school were made on cross sec- tion paper 16"x21" ruled 5 spaces to the inch. Every fifth line was drawn heavier to assist the eye in keeping track of certain points in counting. Along the top of each sheet were written the num- bers corresponding to the numbering of the papers handed in by the pupils examined. There was thus in each instance as many col- umns needed as there were pupils or papers. Every fifth number was written more heavily or in red ink. Along the side at the left were written the numbers of the exercises corresponding to each exercise of the test. In tabulating the scores for Test I, 50 horizontal lines were needed for each school. Every fifth number was written more heavily or in red ink for convenience in counting. There were needed 60 of such large sheets. They were held together by strong steel clips, and pro- tected by heavy flexible card board. Errors only were tabulated. If a pupil made but one error on the test, only one mark was placed in the column assigned to his paper. The other spaces of that column were left blank. An error was indicated by a straight line drawn diagonally at the intersection of the proper vertical and horizontal spaces. The sum of the num- ber of marks for each column gives the total number of errors or score for each pupil. The sum of the number of marks for each row gives the total number of errors made by all the pupils of the school on each exercise. The grand totals of the sums at the right of the page and along the bottom are equivalent. These totals serve as a check in counting. Another convenience in checking was found worth while when the papers were arranged in order of the scores made, beginning at the smallest number of errors. The papers were then numbered consecutively to correspond with the numbers at the top of the large tabulating sheet. In addition to this advantage the median score for any school was found almost at a glance. As a check on the accuracy in tabulating, the number of errors made by each pupil was counted as fast as each paper was disposed of. This total equals the total number of errors recorded at the top of the paper, or the pupil's score on his own paper. Since these two numbers check all along, the sum of the errors made by all Standardizing the Material 31 pupils equals the sum of the errors on each of the exercises, the former number serving as the standard. The tabulation thus is quite rapid and easy so long as the work is close to the numbering at the left of the sheet. As the record of each pupil moves to the right and farther away, the eye strain is increased. The writer overcame this by spacing and numbering the plain side of an ordinary school ruler in duplicate of the column of numbers representing the exercises at the left of the large tabulating sheet. The numbers were written in red ink for prominence and durability. This ruler was moved along column by column right at the space provided for each pupil's record. If certain guiding lines for the ends of the ruler are observed even the tendency to errors is slight and the eye strain is reduced to a minimum. Only after the discovery of these useful devices in tabulating was it pos- sible to dispose of the routine work rapidly and easily. CHAPTER III TENTATIVE SCALES IN THE FORM OF LIST X AND LIST Y From the results of Test II and Test III two tentative scales were made out, and administered as List X and List Y. Exercise number 1 in List Y \vas devised and purposedly added with the idea of trying once more to find an exercise so easy that all pupils could work it successfully. This exercise reads, "The most com- monly spoken language of early New England was Dutch, English, French, Scandinavian, Spanish." Exercise number 14 in List X was added with the purpose of seeing if it would turn out to be a diffi- cult one. This exercise reads, "The gap in location, age, and ideals between the Northern and Southern colonies was bridged over by the founding of Del., Md., N. J., N. Y., Pa." The rest of the content of these two tests is given in full in this chapter. The rank- ing in difficulty was the chief consideration in making the selection for these lists. In case of alternatives, frequency of use in later periods, or social usage determined the choice. The method of administering did not differ in principle from the method used in the preceding experiments but it was written out more in detail for the benefit of the supervisors who assisted the writer in gathering data. The chief purpose in writing up these detailed directions was to try out the plan devised in the preliminary work whereby pupils could assist in marking and correcting their own papers. A copy of these directions for administering the exer- cises to pupils and for scoring the papers is here given: EXERCISES IN UNITED STATES HISTORY Directions to Teachers for administering Lists X and Y: 1. Place upon the blackboard: (a) Columbus discovered Africa, America, Asia, Australia. (b) The Revolutionary War began in the year 1763, 1765, 1775, 1776. (c) The London Company was formed in England, France, Holland, Italy. 2. Say to Pupils: "Here are some exercises in United States History. They are turned into historical facts when a line is drawn around the correct one of the several suggested answers." Draw a line 33 34 Standardized Exercises in History around America, 1775, and England in the examples, and read the e ercises as correct historical facts. 3. "I shall pass around exercises similar to these which are worked in the same way. 4. "Try to work out each exercise whether you are sure of the correct an- swer or not. 5. "Take as much time as you need, but no more, 6. "When all have finished working I shall read the correct answers and you may correct your own papers. 7. "Mark only the numbers of the exercises incorrectly worked. Use an 'X.' " 8. After pupils have made the necessary corrections, direct them to ex- change papers and keep them, 9. Read over the correct answers again and have the pupils check over the papers to detect any errors or omissions in the first correction. 10. Place on the blackboard the numbers corresponding to the exercises. 11. Say to Pupils: "Look on the papers you now have and see if a mis- take was made in working out the first exercise, that is No. 1, If so, please raise your hand and hold it up until I take the count." 12. Count the number of hands raised and place the sum opposite No. l. 13. Repeat the process until the number of errors made on each exercise has been correctly determined. 14. Copy the results for each set of exercises and send them and the papers to Sturgiss B. Davis, School of Education, University of Pittsburgh. List X EXERCISES IN UNITED STATES HISTORY COLONIAL PERIOD Draw a line around the correct suggested answer in each Exercise. 1. When the earliest colonists arrived in America they found the land 1 already inhabited by the Chinese, Cossacks, Indians, Malays, White men. 2. The "Mayflower Compact" was drawn up by the Baptists, 2 Huguenots, Methodists, Pilgrims, Quakers. 3. The "Thirteen Original Colonies" were at one time all under the 3 political control of England, France, Holland, Spain, Sweden. 4. The most common means of travel used by the earliest settlers was 4 by aeroplane, railroad, steamboat, trolley-line, walking. 5. One of the turning points in our Colonial History was marked by 5 the capture of Acadia, Deerfield, Louisburg, Port Royal, Quebec in the French and Indian War. Tentative Scales in the Form of Lists X and Y 35 6. The political control of the Dutch in America came to an end when 6 Governor Stuyvesant surrendered to the English, French, Germans, Spaniards, Swedes. 7. The Quakers founded the colony of Ga., Mass., A'. H., Pa., S. C. 1 8. The first attempt of the English to colonize in America was in what 8 was later called the colony of Conn., Del., Mass., Pa., Va. 9. The period of settlement for most of the colonies took place in the cen- 9 tury beginning with 1400, 1500, 1600, 1700, 1800. 10. Local business affairs in the New England settlements were generally 10 managed by the governor, king, mayor, minister, tov;n- meeting. 11. The territorial claims of the English in America were established by 11 the treaty of Aix-la-Chapelle, America, Paris, Ryswick, Utrecht. 12. The strict Puritan Church of early New England gradually gave 12 way to what has since become known as the Baptist, Congre- gationalist, Lutheran, Methodist, Spiritualist church. 13. The first written constitution in the colonies springing from the 13 people and creating a government was the colony of Conn., Del., Mass., N. J., Fa. 14. The gap in location, age, and ideals between the Northern and 14 Southern Colonies was bridged over by the founding of Del., Md., \. J., A 7 . Y., Pa. Put number of errors here List Y EXERCISES IN UNITED STATES HISTORY COLOKIAL PERIOD Draw a line around the correct suggested answer in each Exercise. 1. The most commonly spoken language of early New England was 1 Dutch, English, French, Scandinavian, Spanish. 2. Among the many hardships of the time were the unexpected attacks 2 by Arabian, Chinese, Cossack, Indian, \egro warriors. 3. Every one of the colonies was founded by men from Africa, Asia, S Australia, Europe, South America. 4. By the time the "Mayflower" had arrived at Plymouth Rock it had 4 crossed the Antarctic, Arctic, Atlantic, Indian, Pacific ocean. 5. Under the "Treaty Elm" William Penn made a lasting peace with S the Canadians, Huguenots, Indians, Negroes, Puritans. 6. The light at night in the early homes came from candles, electricity, 6 gas, kerosene, phosphorus. 7. The Colonists and English soldiers were assisted in the French and Indian War by the loyalty of the Iroguois, Mohegan, Narragan- sett, Pequot, Wampanoag Indian tribe. 36 Standardized Exercises in History 8. To stop the movement of the Colonists westward the French soldiers 8 built a line of forts in western Conn., Mass., Md., N. J., Pa. 9. More lasting and prominent political events took place in Baltimore, 9 Boston, Charleston, Ne K> to s 00 Ox oo --a to to 00 to oo on to V3 On 4*- h- ~ H- > SCHOOLS OO oo ON oo ~J oo ON OO On s 00 * to oo w OO oo $ ON 0? ^J to On to oo 00 On oo ~vl ^1 on ^1 ON ~ o 2 ^4 U) vj to ^O oo oo oo s On 4^ OO OO to NO -vl to O to o vj ^4 OO On 00 oo oo oo tsi t^l oo oo 8 E ON 4*- NO w 8 4* -v| on oo oo 4^ -vj ON oo 00 00 to 00 4^ vj 4- ** oo ^ T! >* to Os to oo oo 4*> to ^0 oo 4- ->4 4^ -v| NO 4^ ^ On 4- o tO .ON NO to ^J ON 4" oo H- to oo ^J to o O ffi to O oo j- 00 00 ON ON -vj ** oo On h- h- - to On OO oo ON 1 I On ^vl s to 3 ON On ^ On NO oo to ON ON to oo (1 4- 4^ to 3 to oo to -^J I* on -vj NO oo oo O j O 1 O H- ^ ft oo 00 oo oo ON oo oo y 00 On oo ON OO to to 4- H- t* S On -vl oo oo ON On to 00 on oo ON On oo -vl oo ON to to s oo 4 vj ON On 00 to ON z s VO oo on to NO On On 00 4- to ON at oo to to NO ON On ON On O O to o 3 4- On 4- 00 On to ^J ON 4>> to O ^4 ON to O >- *n on g NO On ON NO \O % OO on ^1 -v| On On s On -vj 4- 8 to OO O to On ^ 00 to vs*g Q W?^ o &> >t , . s's-S? Tentative Scales in the Form of Lists X and Y 39 _j i| a NUMBER OF EACH EXERCISE \ TABLE IV SHOWING HOW DATA WAS RECORDED BY SCHOOLS NO 00 - en 4- H to - * W to "^ s t On on to to to to to oo 00 00 00 to H- to > SCHOOLS i On ON 00 OO X oo * oo 4- 5 4- 4- H- ~ 4, o w -X> yo S 3 s oo S 4- 00 to to to S o vo o Oi S 3 vo oo S oo to to to ON to to O to oo ON VO to -vj VO oo ON s On 4- to oo oo ~ vo - M ON t 5 oo OO + On oo On vo VO 00 00 to " OO to OO 2 * ON ON H- to to o oo ^ to *-* to 3N ON ON on VO ON oo N VO oo ^ ~ a S ^ ~ to* M O to h- ~ 4- to 00 oo 4- to to g oo oo s to to to oo oo ON on, vo ~ on to ON OO - S ol to ON to oo to oo oo On 4, O * $ On 3 00 to to to H _ 1 M ^ 1 > H- to r 3 s d i on ON oo oo oo to On On oo vo On 4- on 2 to to OO S 00 to to On On s 4- On oo to to on VO ON K * 00 to vo to oo ts to S ON on M< to O s On a On On to K to ON oo + to A) i On On o O VO On On 4*- 4^ On to oo to On 3 to oo OO oo to oo OO S O &j "i ~~ " "'* V5 O 40 Standardized Exercises in History TABLE V TOTAL NUMBER OF ERRORS MADE ON EACH EXERCISE IN LIST X Exercise Number Number of Errors Same Exercise in Test II Same Exercise in Test III Grand Total Percentage Value Sigma Value 1 44 6 50 2.42 .63 2 49 13 62 3.00 .70 3 48 '36 84 3.70 .79 4 82 64 146 6.44 1.03 5 257 - 257 16.42 1.55 6 230 '84 314 15.20 1.50 7 400 183 583 25.71 1.87 8 574 173 747 36.15 2.16 9 520 200 720 34.85 2.13 10 750 428 1178 51.95 2.53 11 857 285 1142 50.36 2.50 12 986 571 1557 68.66 2.97 13 940 347 493 1780 64.44 2.85 14 956 956 61.09 2.77 Total No. of Pupils taking each test 1564 500 702 TABLE VI TOTAL NUMBER OF ERRORS MADE ON EACH EXERCISE IN LIST Y Exercise Number Number of Errors Same Exercises in Test II Same Exercises in Test III Grand Total Percentage Value Sigma Value 1 140 140 8.95 1.19 2 38 '27 65 2.87 .69 3 72 51 123 5.42 .95 4 88 60 148 6.53 1.04 5 123 123 7.86 1.13 6 177 193 370 16.32 1.54 7 325 223 548 24.16 1.82 8 512 Io7 679 32.86 2.07 9 496 384 880 38.81 2.23 10 754 754 48.18 2.47 11 795 504 1299 57.29 2.67 12 950 314 1260 60.98 2.76 13 1107 487 1594 70.30 3.02 14 1155 363 550 2068 74.86 3.15 Total No. of Pupils taking each test 1564 500 702 Tentative Scales in the Form of Lists X and Y TABLE VII RANK OF EXERCISES IN LIST X IN SECOND TRIAL 41 Rank and No. of the Exercise Percent of all Pupils Missing Sigma Value Rank in Second Trial Abstract of Contents of Each Exercise 1 2.81 .68 I Indians here before the white men came 2 3.13 .72 III "Mayflower Compact" for the Pilgrim Fathers 3 3.07 .71 II England controlled 13 original colonies 4 5.24 .93 IV Walking most common means of travel 5 16.42 1.55 V Capture of Quebec turning point 6 14.70 1.48 VI Stuyvesant surrendered to the English 7 25.56 1.86 VII Quakers founded Pennsylvania 8 36.68 2.18 IX Va. colony first English settlement 9 33.23 2.08 VIII Period of settlement from 1600 to 1700 mostly 10 47.93 2.46 X Townmeeting managed business affairs 11 54.76 2.60 Xi Treaty of Paris established English claims 12 63.00 2.82 XIV Puritan church represented by Con- gregationalist 13 60.07 2.74 XII First written constitution in Conn. 14 61.09 2.77 XIII Pa. bridged gap between North and South Correlation by Spearman's "Footrule" is .98. TABLE VIII RANK OF EXERCISES IN LIST Y IN SECOND TRIAL Rank and No. of the Exercise Percent of all pupils Missing Sigma Value Rank in Second Trial Abstract of Contents of Each Exercise 1 8.95 1.19 V English Language most common 2 2.43 .63 I Indian attacks among the hardships 3 4.60 .88 II Men from Europe founded the colonies 4 5.62 .97 III The Mayflower crossed the Atlantic Ocean 5 7.86 1.13 IV The Treaty Elm marked peace with Indians 6 11.31 1.32 VI Candles furnished the light at night 7 20.77 1.71 VII Iroquois assisted the English soldiers 8 32.72 2.07 IX French built a line of forts in Pa. 9 31.69 2.04 VIII Philadelphia a political center 10 48.18 2.47 X Huguenots settled Charleston 11 50.80 2.50 XI Last colony founded in 18th century 12 60.70 2.76 XII Braddocks Defeat in French and Indian War 13 70.74 3.03 XIII "United Colonies of New England," in 1643 14 73.80 3.12 XIV Lord Delaware, Governor of Va. Correlation by Spearman's "Footrule" is .97. CHAPTER IV WEIGHTING THE RESULTS OF THE DATA ON EACH EXERCISE SEC. 1. Plan of Procedure. After tabulating the total number of errors made by all the pupils taking each exercise these numbers were transformed into percentages. The first question answered was : what per cent, of all the pupils examined failed in exercise number one, two, three, four, and so on for the whole list? The plan adopted for doing this was as follows: a blank book of cross section paper was procured which had more than twice as many pages as the number of exercises to be weighted. The right hand pages of the book were numbered consecutively in large figures in red ink. As many pages were numbered as there were exercises to be weighted. In a column at the left on these pages were listed the numbers of the large tabulation sheets in which the data from each school were tabulated. After each of these numbers was listed the total number of errors found for each exercise on the large tabu- lation sheet. The sum of these numbers taken sheet by sheet gave the total number of errors made on any given exercise. On the left hand pages of the book were made the necessary calculations to determine the weight of each exercise. These calcu- lations consisted in finding the percentages and then transmuting them into sigma values. When the data on the 1,250 pupils tested in 1918 on Test I were utilized, advantage was taken of the con- venient quotient given when 100 is divided by 1250. This is .08 and was used as a constant multiplier in finding the percentage of errors made on each exercise. The work was further facilitated by the use of Crelle's Calculating Tables. An allustration is offered to save further description. On a certain exercise, X pupils failed out of the 1,250 examined. Required, to find the percentage of failure in this instance. X 100 100 X - X - = - X = 1250 1 1250 1 When the equation is stated in this form it is evident that the frac- tion 100/1250 is a constant in each calculation, and the variable 43 44 Standardized Exercises in History is always the number of errors made on each exercise. Hence the calculation in each instance is reduced to multiplying .08 through means of a calculating table by the total number of errors made on each exercise. The next step was the transmutation of these percentage values into sigma values. This was done by means of the tables prepared for such a purpose by Rugg. Both Tables V and VI in this man- ual were considered but Table V was chosen because it is built upon the usual five divisions of the base line of the Normal Curve of Frequency. It is also accurate enough for the purposes of this study. No attempt is made here to explain the Normal Curve of Fre- quency or its use in finding the absolute ease or difficulty of ques- tions. Its use is now widely recognized and accurate descriptions and explanations are available to students of educational problems. Many investigators of educational problems have used the P. E. values in weighting questions. Sigma values were used in this study because these new tables of Rugg are more convenient to read and the values may be found directly regardless of the median point of reference. The sigma unit value was also used because it is a nat- ural function of the Normal Curve and is widely used by students in Biology, Sociology, Agriculture, and Actuarial Work. For the sake of comparison, however, the P. E. values of Test I were calculated and compared with the sigma values. In order to get these P. E. values it was necessary to assume a point of reference, since no exercise was found which all pupils could answer. This point of reference was 4.3 P. E. from the mean ordinate. It was derived from the first exercise which 98.68% of the pupils worked out correctly. Table XLVII in Buckingham's Study of "Spelling Ability" was used in calculating the P. E. values. Table IX of this study shows the comparisons in full. SEC. 2. Tests for the Reliability of the Data. Several means of checking on the reliability of data were considered before the values here presented were considered valid. These means are all discussed in books on Statistical Method and need only to be men- tioned here. The curve of errors shown in Fig. I was plotted and compared with the normal curve drawn from the same median ordinate. Weighting the Results TABLE IX COMPARISON OF P. E. AND SIGMA VALUES 45 No. of Percent Difference P. E. Dist. in P. E. Sigma Exercise Correct between 50% Values P. E. above Value Value and Score Zero Point 1 98.68 -48.68 -3.3 1.00 10 4 2 68.00 -18.00 - .693 3.607 36 20 3 64.08 -14.08 - .535 3.765 37 22 4 63.08 -13.08 - .496 4.104 41 24 5 55.20 - 5.20 - .194 4.406 44 24 6 55.84 - 5.84 - .216 4.084 40 24 7 65.4 -15.4 - .588 3.712 37 21 8 20.04 +29.96 + 1.243 5.543 55 33 9 74.96 -24.96 - 996 3.004 30 19 10 52.20 - 2.20 - .082 4.218 42 25 11 55.04 - 5.04 - .187 4.113 41 24 12 85.64 -35.64 -1.576 2.724 27 15 13 19.04 +30.96 + 1.296 5.596 55 + 34 14 81.20 -31.20 -1.313 3.287 29 16 15 29.84 +20.16 + .782 5.082 50 30 16 44.88 + 5.12 + .187 4.487 45 26 17 38.72 + 11.28 + .426 4.726 47 28 18 59.08 - 9.08 - .341 3.959 39 23 19 68.44 -18.44 - .710 3.590 35 20 20 47.36 + 2.64 + .097 4.397 44 26 21 60.20 -10.2 - .383 3.019 39 23 22 87.64 -37.64 -1.713 2.587 25 + 14 23 74.64 -18.64 - .719 3.581 35 + 19 24 86.84 -36.84 -1.656 2.644 26 14 25 90.16 -40.16 -1.980 2.320 23 12 26 29/2 + 20.68 + .807 5.107 51 30 27 38.84 + 11.16 + .422 4.722 47 28- 28 71.52 -21.52 - .842 3.458 34 19+ 29 55.76 - 5.96 - .213 4.087 41- 24- 30 39.80 + 10.20 + .383 4.683 47 27 31 58.28 - 8.28 - .311 3.989 40- 23 32 55.40 - 5.4 - .201 4.099 40+ 24 33 18.92 +31.08 + 1.307 5.607 56 34 34 37.56 + 12.44 + .468 4.768 48 28 35 56.92 - 6.92 - .258 4.042 40 23 36 68.16 -18.16 - 69S 3.602 36 20 37 20.08 +29.92 + 1.243 5.543 55 33 38 96.80 -46.80 -2.746 1.554 15 7 39 42.92 + 7.08 + .261 4.561 45 26+ 40 72.00 -22.00 - .864 3.436 34 19 41 68.52 -18.52 - .714 3.586 36- 20 42 49.28 + .72 + .026 4.326 43 25 43 86.88 -36.88 -1.663 2.637 26 14 44 82.84 -32.84 -1.403 2.897 28+ 16- 45 87.12 -37.12 -1.677 2.623 26 14 46 59.32 - 9.32 - .349 3.951 39 23 47 53.68 - 3.68 - .172 4.128 41 24 48 21.76 +28.24 + 1.155 5.455 54 32+ 49 60.80 -10.80 - .407 3.895 39 22 50 98.30 -48.15 -3.077 1.223 12 5 46 Standardized Exercises in History The same curve of errors was smoothed successively until no appreciable change was evident. In both these instances the theoretical curve did not differ de- cidedly from the real curve obtained from the responses of children. The check on the reliability of the data which was most used was the consistency shown in the percentage of errors pupils made on each exercise from school to school. In the case of each test there came a point where the addition of more data from pupils made little or no change. When that point was reached, no further data were taken. Another test of the reliability of the data, though an indirect one, was the constant tendency revealed in the results from each school. This tendency was the wide variability of the achievements of individual pupils. Since the pupils who took the test were all the most advanced ones in the elementary school such variability was not expected, unless it was due to inherent mental differences which the history tests brought out. These statements are supported by Table X which shows the wide range of errors for each of the 58 schools taking Test I. If the variability was due to the inherent capacity of pupils to master the formal requirement of the school in United States History, the scores in these history exercises should indicate corre- lation with the scores of a recognized intelligence test. Table XI was prepared from the scores of 203 pupils who had taken the Pressey Intelligence Test and Test III of the history exercises. The scores are not altogether comparable since the intelligence test is really a composite of several tests, each intended to test a different mental ability. The history test on the other hand invokes prob- ably largely the factors of selective judgment and recall. How- ever, it was the best data obtainable at the time on this point. An examination of the correlation table made up of these two classes of scores shows that they produced much the same general effect. In both tests the variability was marked, but in general the brighter pupils made the better history scores. The median score in the intelligence test was 140.6. Only twelve pupils ranking below this median score in intelligence made a high score in the history test. Both tests also showed large individual differences in Weighting the Results TABLE X RANGE OF ERRORS MADE BY INDIVIDUAL PUPILS IN THE SCHOOLS TAKING TEST I 47 Schools Range Schools Range 1 1-27 XXX 13-33 II 4-35 XXXI 5-27 III 8-34 XXXI i 10-36 IV 11-33 XXXIII 2-28 V 17-31 XXXIV 7-28 VI 9-41 XXXV 4-33 VII 6-32 XXXVI 1-30 VIII 6-37 XXXVII 8-40 IX 5-32 XXXVIII 11-31 X 6-31 XXXIX 11-31 XI 5-30 XL 5-29 XII 7-26 XLI 7-29 XIII 5-25 XLII 9-37 XV 4-29 XLIII 9-36 XV 5-27 XLIV 7-13 XVI 7-25 XLV 2-32 XVII 11-33 XLVI 8-32 XVIII 13-36 XLVII 7-35 XIX 5-29 XLVIII 2-40 XX 7-27 XLIX 2-30 XXI 6-23 L 6-31 XXII 7-31 LI 9-36 XXIII 8-32 LII 4-38 XXIV 9-36 LI II 12-36 XXV 17-40 LIV 1-18 XXVI 7-23 LV 7-35 XXVII 3-34 LVI 12-33 XXVIII 6-38 LVII 3-36 XXIX 11-38 LVIII 16-37 the pupils. These facts suggest that the material of the history test contained exercises, some of which were inherently easy and some inherently difficult. This last statement is consistent with the con- stant tendency revealed in the percentage of errors pupils made on certain exercises when the attainments of pupils were compared by schools. Having found the marked differences in the scores of pupils of the same school it was pertinent to find out whether the same char- acteristic would be found among the different schools themselves. The measure used for this test of reliability was the coefficient of dispersion based upon the semi-inter-quartile range and the median score found for each school taking Test I. Table XII shows these 48 Standardized Exercises in History POINTS IN PRESSY INTELLIGENCE TEST /;/! ;/ v !/ : !/ Table XI. Correlation table showing relationship between scores correct in Pressy Intelligence Test and Test III of the Exercises in Colonial History coefficients of dispersion for each of the 58 schools taking the test. The difference noted in the range is inconsequential when the facts are known about these two schools. The school in which the lowest coefficient was found is one whose scope is limited and the super- visor is a typical drill master. The school in which the highest coefficient appeared is located in an industrial center in the North- west. The pupils of this school were expecting to take a commer- cial course the following year in the high school. What is striking about this table however is the range of the middle-fifty per cent, of the cases. It is less than 10 points; Q is approximately 4.8; and the C. D. is but .22. These findings are consistent with the state- ment given out by the New Committee of Eight on courses in his- tory. This statement is to the effect that the present content of American History courses for the Seventh and Eighth Grades is the best standardized of all courses in history. The coefficient .22 con- forms remarkably well with the index of variability established by Courtis for his test in the four fundamentals of arithmetic. His Weighting the Results 49 standard was derived in approximately the same way and is given as .20. No question has ever been raised either as to the content or value of the data upon which Courtis derived his standard. In concluding this topic of variability the writer feels justified in saying that the content of the history exercises is well recognized material; and that the form of the exercises always reveals indi- vidual differences in the same class or school. When large groups are compared, however, the performance of pupils is approximately uniform. Table XII COEFFICIENTS OF DISPERSION DERIVED FROM THE SCORES OF THE 58 SCHOOLS TAKING TEST I Index Frequency 9 1 11 2 12 4 13 2 14 2 Range = 9 40 15 4 16 2 Median = 21.7 17 4 18 1 Q. = 4.8 19 1 20 3 C. D. = .22 21 4 22 4 23 6 24 1 25 4 26 2 28 1 30 4 31 1 33 1 34 2 37 1 40 1 Total 58 50 Standardized Exercises in History g Ec . 3. The Reading Difficulty of the Exercises. In evaluating the reliability of pupils' responses the reading difficulty must be taken into consideration. Can the assumption be maintained that pupils may know history but because of the difficulty of the form of the test itself they do not know what is wanted of them? Some investigators have dismissed this question by saying that if pupils cannot read their school texts by the time they have reached the eighth grade, they are helpless so far as formal schooling goes. If the words used in the test are familiar to pupils, and if the con- struction of sentences are not unusual, the claim of these investigators is logical. In order to get direct evidence on the reading difficulty of Test I the writer submitted the exercises one by one to individual pupils. He went to the principal's office in two different schools and selected children from the eighth grade at random for personal interviews. In each instance the test was put before the pupil with the request that he read the exercise and simply state just what was to be done after reading it. No answers were required, and no comment made. The aim of the examiner was to get the pupil to think out loud. A case record was kept for each pupil by noting any peculiarity or difficulty in his responses. In no instance was there a single pupil who did not know exactly just what was wanted in each exercise. The words delusion, influential, indentured, and professional were not fully comprehended by some pupils when they were questioned after the examination, but in each case these same pupils gathered from the context of the exercise exactly what was wanted. These pupils had an impressionistic knowledge of these difficult words and recognized them sufficiently to use them in sentences. In the case of the exercise referring to the "witchcraft delusion occurring among the Puritans," some pupils passed over the word delusion and asso- ciated witchcraft with Puritans at once. Instances were found where the pupil could work out this exercise but could not use the word delusion readily in a sentence. The examination in this case was conducted somewhat after the manner of Terman in the vocabulary test used in his Stanford Revision of the Binet-Simon Test for Intelli- gence of School Children. Another instance in line with the above came out in respect to the word, indentured. One pupil remarked when questioned Weighting the Results 51 about its use in the exercise, "You do not have to know what it means exactly; servants and slaves gives you the idea of what is wanted/' This boy knew the correct answer to the exercise also. These private interviews with pupils threw much direct light on the reading difficulty of these particular exercises and indicated that the mental process involved was associative memory to a marked extent. In order to get further data on the reading difficulty of this type of testing the scores of 241 eighth grade pupils who had taken the Monroe Reading Test, and Test III of the History exercises, were compared in a correlation table. Table XIII reveals at a glance that the degree of correspondence is low. As a matter of fact it is so low that it is negligible, being .04 by the Pearson formula. Test I could have been improved by simplifying the wording. However, it was thought wise to leave the exercises in the original form for the purpose of this study. All the words in the test are in use in standard text books and teachers have not criticized their inclusion. SCORES IN HISTORY TEST III 7 10 n 'J lt> '7 /% ' \ j~/fit i. I TABLE XIII Showing Correlation Between Scores in Monroe Reading Test and Test III in History 52 Standardized Exercises in History Lists X and Y represent an improvement in the wording of exercises. Through practice one can readily become skilled in fram- ing exercises that meet all the standards set up in Chapter I. In general it may be offered that so long as the vocabulary of the exer- cises correspond to that of standard text books, this reading difficulty of words, or sentence structure, does not seem to be at fault nearly so much as the lack of purpose in reading. This was shown very definitely in a reading test prepared by the writer in which para- graphs were taken verbatim from a school history which the pupils had used. SEC. 4. The Element of Success by Chance. Formula. In addition to the reading difficulty it was suggested that some pupils were successful through the operation of a chance selection of the correct one of the several suggested answers. The tabulation of the scores on the large tabulation sheets where all the scores from a single school could be seen at a glance, gave no evidence of the operation of the law of chance. The difficult exercises were worked out only by the pupils who made good scores consistently throughout the test as a whole. When pupils were questioned individually regarding the reasons for the answers chosen, they always gave a plausible explanation even though their answers were incorrect. For the pur- poses of diagnosis it was desired that pupils select what they consid- ered to be the correct answer, and they were encouraged to try each one of the exercises. In no instance w r as there found evidence of wild, haphazard guessing at the correct answer among pupils who took the testing material for it was all more or less familiar to them. They thought they knew whether they did or not. In the event that it seemed advisable at any time to make cor- rections for the element of chance entering into the scores of pupils, a correction formula was developed. The formula was suggested by the one offered by McCall which he used in eliminating the effects of chance selection in his experiment with the "True and False" type of of examination. This is given in an article entitled "A New Kind of School Examination." The formula offered in this instance was: "Pupil's Score (number correct) (number wrong." Starting with this formula it was developed so that any number of chance selections could be considered. Simultaneously Weighting the Results 53 Buckingham did the same thing and expressed his formula as follows: "S = c where S is the score after account has been n 1 taken of the elements of chance ; c is the number of correct answers ; w is the number of wrong answers; and n is the number of alter- natives offered for each question." This formula is especially convenient for use in correcting scores where the number of correct results are always wanted. The fol- lowing formula is more convenient where the test is long and the number of errors only are considered as was done in the case of this study : True Score = n c (w) c 1 where, n = the number of exercises in the test c = the number of chances in each exercise of guessing the right answer w = the number of wrong answers To illustrate: Suppose a test contained 15 exercises. There were 5 suggested answers in each exercise. A certain pupil made 12 errors on his paper. The answers given by the pupil were inconsistent with his training. It is required to find his probable true score. Substituting the numbers 15, 5, and 12 in the formula, True score = 15 -- *,(12) = Instead of the pupil being credited with a score of 3 correct, a truer estimate of his real knowledge on this test is probably zero. To show how nicely this formula works out when the true status of conditions is known, the following additional illustration and analysis will make the point clearer. Suppose a test contains 20 exercises. In each exercise there are 5 chances of selecting the correct answer. I know the answers in 5 different exercises for certain. I am ignorant entirely of the rest but take a chance at hitting the right ones. When my paper is returned to me I find that the examiner has credited me with 8 answers correct. Now a score of 8 is absurd, for I knew only five answers. The 54 Standardized Exercises in History extra three credits came to me by mere chance. A score of five is all I deserve; for, 20 = the number of exercises in the test. 5 = the number of answers really known. 15 = the number of answers not known at all. 3 = the number of answers possible to get correct by law of chance. 5 + 3 = total number of answers correct by knowledge and by chance. 20 8 = 12, the number marked wrong by the examiner. True Score = 20 --(12) = 5, which expresses exactly the score I deserve. CHAPTER V CLASSIFICATION OF ALL THE STANDARDIZED EXER- CISES INTO SCALES C AND D AND TESTS E AND F Since the data gathered from Tests I, II, III, and Lists X and Y stood the proposed tests of reliability, all the 85 exercises were arranged in order of their value in terms of the sigma unit. This arrangement is given in Table XIV. A study of this table reveals the possibility of arranging the different exercises into scales and tests based upon their difficulty. Two scales and two tests are here proposed. Other arrangements are possible, but the ones given illustrate the theory of procedure. The organization of the material for Test E will be considered first. In constructing a standardized test the aim should be to include those points whose difficulty is as nearly equal as possible. By ref- erence to Table XIV it may be seen that that largest number of exercises which approximate the same value lie between the sigma values of 2.03 and 2.55. On each side of these points in the dis- tribution, the values do not run so closely together in such a large number of cases. These 32 exercises may be set aside tentatively as the possibility of a standardized test. On further examination of the remaining exercises the possi- bility of a scale seems evident. The word scale is here used in the sense of a series of exercises whose values differ equally from one another by some adopted unit. By moving the decimal point in the sigma values one place to the right, and then forcing the num- ber to the next unit if the first decimal figure is greater than .5 and conversely neglecting the decimal figure if equal to or less than .5, a series of 16 values may be found which differ from one another by two points. Such a series of exercises is offered in the form of Scale C. The actual sigma values and approximations are given in the score sheet for Scale C. On further examination of the distribution of the sigma values, the possibility of another scale is seen if a unit of .25 is taken as the basis. Such a scale is offered in the series of exercises in the form of Scale D. The values for the proposed scale are given in the 55 56 Standardized Exercises in History score sheet for Scale D. The guide to forming this scale was the succession of steps running from .50 up to 3.25. The 12 exercises presented approximate these steps quite closely. Table XIV THE ABSOLUTE VALUES OF ALL THE EXERCISES ARRANGED IN ORDER VALUE ABSTRACT OF CONTENT .43 The Mayflower was a ship. .54 The White men found the Indians here. .64 The Pilgrims came from England. .69 Sudden attacks from the Indians were among the hardships. .70 The Mayflower Compact was drawn up by the Pilgrims. .73 First houses were generally built of logs. .79 England once controlled all the colonies. .95 Men from Europe founded the colonies. .03 Walking was the most common means of travel. .04 The Mayflower crossed the Atlantic ocean. .13 The Treaty Elm marked a lasting peace with the Indians in Pa. .19 English was the most commonly spoken language. .20 The colonies increased to 13 by the time of the Revolutionary War. 1.22 Pocahontas was an Indian girl. 1.24 Pocahontas is said to have saved the life of John Smith. .37 New York was founded by the Dutch. .40 By the latter part of the 18th century democracy was wanted. .41 The City of Brotherly Love was founded by William Penn. .41 The "Great Treaty" was between t^e Indians and William Penn. .46 The Charter Oak was a tree. .50 The Dutch colony surrendered to the English. .54 Candles furnished the light in the early homes. .55 The capture of Quebec was a turning point in American history. .58 The French and Indian War was the most impo'rtant to the colonists. .64 The English were the most numerous of all the nationalities. .82 The Iroquois Indians assisted the English against the French.. .85 Corn was the principal native food crop. .86 Tobacco was the main export crop from Va. .87 The Quakers founded Pa. .87 In the French and Indian War Washington gained experience for the Revolution. .94 Emigration was generally westward. .95 The Plymouth Rock is in Mass. 2.03 Stuyvesant was the last of the series of Dutch governors. 2.04 Massasoit kindly received the Pilgrims. 2.04 The Act of Toleration was decreed by the Catholics in Md. Scales and Tests 57 2.05 Miles Standish led in fighting the Indians. 2.05 Jamestown was founded in 1607. 2.07 The French built a line of forts in western Pa. 2.12 The Witchcraft Delusion occurred among the Puritans. 2.13 The period of settlement was from 1600 on approximately. 2.16 Roger Williams founded R. I. 2.16 The first attempt of the English to colonize was in Va. 2.23 Philadelphia was the seat of permanent political events. 2.26 Slavery was introduced in 1619. 2.28 The Quakers were persecuted by the Puritans. 2.29 The Holy Experiment was tried out by William Penn. 2.29 Geo. Oglethorpe founded Ga. to help poor debtors of England. 2.31 Indentured servants held in Va. 2.34 First settlement of Catholics in Colony of Md. 2.37 First Pilgrims landed in 1620. 2.37 First college founded was Harvard. 2.38 Sir Edmund Andros was the Royal Governor of New England. 2.38 William Pitt the most outspoken friend in the English Parliament. 2.38 Thomas Hooker led in founding Conn. 2.39 Bacon's Rebellion was against Lord Berkeley. 2.42 The Spanish once invaded Ga. 2.44 The Patroons were Dutch landowners. 2.45 New Sweden later became the colony of Delaware. 2.46 Mason and Dixon line surveyed between Pa. and Md. 2.47 Huguenots founded Charleston.' 2.50 Treaty of Paris established the English claims in America. 2.51 The Germans settled mostly in Pa. 2.53 The townmeeting settled local business affairs in New England. 2.55 First settlement of the Swedes was in Delaware. 2.61 Preachers the most influential professional class in New England. 2.66 Ga. was the last colony founded. 2.67 The last colony was founded in the 18th century. 2.74 William Claibourne led a rebellion against Lord Baltimore. 2.76 Braddock's Defeat occurred in the Fourth Inter-Colonial War. 2.77 The founding of Pa. bridged the gap between the North and South. 2.80 William Bradford was once leader of the Pilgrims. 2.85 Conn, had the first written constitution springing from the people. 2.92 John Winthrop was the leader in founding Boston. 2.97 The Congregational ist Church grew out of the Puritan Church. 3.01 The government of early Pa. was proprietary. 3.02 The United Colonies of New England was formed in 1643. 3.03 Rhode Island had the charter form of government. 3.05 The right of woman's suffrage was granted for a time in New Jersey. 3.15 Lord Delaware resided in England while governor of Va. 3.32 Boston News Letter the first newspaper in America. 58 Standardized Exercises in History 3.32 John Berkeley and Geo. Carteret once owned N. J. 3.35 John Locke drew up the Grand Model. 3.36 1619 marked the time of the first Representative Assembly in A. 3.39 Gorges once proprietor of Maine. 3.45 The Huguenots' settlement at New Rochelle was quite prosperous. Since Scale D is comprised of only 12 exercises its construction may be used to show how scales may be graphically illustrated. Figure 3 was drawn for this purpose and the explanation of it is as follows: Let the line A B represent the base line of the normal probability curve broken off at 2.5 on each side of the mean ordi- nate. This length is equivalent to a base line of 5 divisions with O at one end and 5 at the other. If each of the sigma units be divided into 100 equal parts, the location of any sigma value is easily determined when the graph is drawn on cross section paper ruled 10 lines to the inch. Since the values of Scale D extend approximately from .50 to 3.25 with a unit difference between any points of .25 , the location of the 12 exercises is readily apparent. In the absence of an exercise of zero and .25 difficulty the first exercise must be placed at .5 and the last at 3.25 . Exercises 3, 7, and 11 fall at 1(7, 20 and 30 respectively. The intervening values fall at the successive quarter points as illustrated in figure 3. All the values of the 85 exercises given in table XIV may be located on a base line divided and subdivided as the line A B. In the case of Test E, described more fully below, all the 26 values fall within less than half the distance between two sigma points. In the formation of these two scales, values are needed extend- ing throughout nearly the whole range of the distribution of sigma values. Six out of the 32 exercises set aside tentatively to form a test were used in the formation of Scales C and D. The 26 remain- ing exercises out of the 32 set aside were used to make up what is A.. . . , / . J. . ? . 1 A .->. . T. . ?_& X $ . , _____ J3 FIG. 3 SHOWING THE LOCATION OF EACH OF THE 12 EXERCISES OF SCALE D. Scales and Tests 59 here presented as Test E. The values for each of the exercises in this test are given in its accompanying score sheet. By reference to it one sees the greatest possible difference between any two exercises is less than .50 sigma. When the values of the exercises are grouped into a series with a class interval of .05, the average sigma value of the series is found to be 2.29; the standard deviation from this central tendency is but .15, and the coefficient of dispersion based upon the average and standard deviation is only .068. These sta- tistical methods were applied to this series of exercises to test the homogeneity of the list. After Scales C and D and Test E had been formed, there remained 31 exercises whose values did not conform either to the definition of a true test or a true scale. These exercises are just as valuable for testing purposes though not so convenient to use. Their value for the diagnosis of a pupil's difficulties is even better, how- ever, than the other proposed measures since the range of the weight- ings is greater. These 31 exercises were then arranged in order of value with a score sheet and is here given as Test F. Test F is an illustration of a popular conception of the word test; that is, a series of questions or exercises whose value is simply known. In grading a pupil's work credit should be given on each point earned in proportion to its absolute value. Test E is an illustration of a true test wherein the value of all the questions or exercises is the same or approximately so. A pupil's score on such a test equals the sum of the points earned multiplied by the value common to each question or exercise. In a true scale the questions or exercises are arranged in order of the values, the easiest usually of zero difficulty being placed at the head of the list. As a pupil progresses in learning he is able to work farther down the list. His score is represented by the value of the last exercise worked successfully. Theoretically the effort required to progress from one point to the next is the same since the interval between any two points is a constant value. The words test and scale have been much used interchangeably. The distinction here given is offered to prevent a misconception of the two scales and two tests presented in this chapter. 60 Standardized Exercises in History SCALE C 1. The Mayflower was a chapel, hall, hotel, plant, queen, ship. 2. The Pilgrim Fathers came from England, France, Germany, Spain, Sweden. 3. The ''Thirteen Original Colonies" were at one time all under the politi- cal control of England, France, Holland, Spain, Sweden. 4. The most common means of travel used by the earliest settlers was by aeroplane, railroad, steamboat, trolley-lines, walking. 5. Pocahontas is said to have saved the life of Sir William Berkeley, Jona- than Edwards, William Penn, John Rolfe, John Smith. 6. The City of Brotherly Love was founded by William Berkeley, George Calvcrt, John Endicott, William Penn, John Smith, Miles Standish. 7. Of the nationalities represented in all the colonies the most numerous were the Dutch, English, French, Germans, Irish, Scotch, Swedes. 8. The principal native food crop was barley, corn, oats, potatoes, rice, rye, wheat. 9. The Toleration Act in Md. was decreed by the Catholics, Episcopalians, Puritans, Quakers. 10. More lasting and prominent political events took place in Baltimore, Boston, Charleston, New York, Philadelphia than in any other one city. 11. The Patroons were Dutch fishermen, fur traders, landowners, miners, preachers, teachers. 12. The most influential professional class in New England were the editors, lawyers, physicians, preachers, teachers, writers. 13. The first written constitution in the colonies springing from the people and creating a government was the colony of Conn., Del., Mass., N. J., Va. 14. The Charter form of government was enjoyed by the people of Del., Ga., Md., Pa., R. I., Va. 15. Lord Delaware lived in England nearly all the time he was governor of Conn., Del., N. J., R. I., Va. 16. The First Representative Assembly held in America was in 1607, 1619, 1620, 1643, 1754. Scales and Tests 61 SCORE SHEET FOR SCALE C No. of Each Exercise The Correct Answer Approximate Value Sigma Value Suggested Practical Value 1 Ship 4 .43 1 2 England 6 .64 1 3 England 8 .79 1 4 Walking 10 .03 1 5 John Smith 12 .24 1 6 William Penn 14 .41 1 7 English 16 .64 2 8 Corn 18 .85 2 9 Catholics 20 2.04 2 10 Philadelphia 22 2.23 2 11 Land Owners 24 2.44 2 12 Preachers 26 2.61 3 13 Connecticut 28 2.85 3 14 Rhode Island 30 3.03 3 15 Virginia 32 3.15 3 16 1619 34 3.36 3 62 Standardized Exercises in History SCALE D 1. When the colonists first landed in America they found the natives to be Chinese, Indians, Malays, Negroes, White Men. 2. The houses of the first colonists were generally built of brick, cement, dirt, logs, marble. 3. Every one of the colonies was founded by men from Africa, Asia, Aus- tralia, Europe, South America. 4. Pocahontas was an English, French, German, Indian, Spanish girl. 5. One of the turning points in our Colonial History was marked by the capture of Acadia, Deerfield, Louisburg, Port Royal, Quebec in the French and Indian Wars. 6. The Colonists and English soldiers were assisted in the French and Indian Wars by the loyalty of the Iroquois, Mohegan, Narragan- sett, Pequot, Wampanoag Indian tribe. 7. The last of the Dutch governors was Hudson, Minuit, Stuyvesant, Win- throp, Yeardley. 8. The Quakers in Mass, were persecuted by the Baptists, Dutch Re- formed, Methodists, Moravians, Puritans. 9. The first settlement by the Swedes was in Conn., Del., Ga., Md., Mass., N. C., N. H., N. Y. 10. William Bradford was once Leader of the Baptists, Catholics, Pilgrims, Quakers. 11. The "United Colonies of New England" was formed in 1634, 1643, 1683, 1733, 1774. 12. The first newspaper in America was the Boston News Letter, New York Times, Philadelphia Ledger, Poor Richard's Almanac, Providence Journal. Scales and Tests 63 SCORE SHEET FOR SCALE D No. of Each Exercise The Correct Answer Approximate Value Sigma Value Suggested Practical Value 1 Indians .50 .54 1 2 Logs .75 .73 1 3 Europe 1.00 .95 1 4 Indian 1.25 1.22 1 5 Quebec 1.50 1.55 2 6 Iroquois 1.75 1.82 2 7 Stuyvesant 2.00 2.03 2 8 Puritans 2.25 2.28 2 9 Delaware 2.50 2.55 3 10 Pilgrims 2.75 2.80 3 11 1643 3.00 3.02 3 12 Boston News 3.25 3.32 3 Letter 64 Standardized Exercises in History TEST E 1. The Pilgrims were kindly received by Chief Canonicus, Massasoit, Philip, Poiuhatan. 2. Miles Standish led in fighting the Dutch, Indians, Puritans, Swedes. 3. Jamestown was founded in 1607, 1619, 1620, 1643, 1733, 1754. 4. To stop the movement of the Colonists westward the French soldiers built a line of forts in western Conn., Md., Mass., Pa., N. J. 5. The Witchcraft delusion occurred among the Baptists, Catholics, Dutch, Indians, Puritans. 6. The period of settlement for most of the colonies took place in the cen- tury beginning with 1400, 1500, 1600, 1700, 1800. 7. Roger Williams founded* the colony of Ga., Md., Mass., N. C., N. J., Pa., R. I., S. C., Va. 8. The first attempt of the English to colonize in America was in what was later called the colony of Conn., Del., Mass., Pa., Va. 9. Slavery was first introduced in 1607, 1619, 1620, 1643, 1683, 1775, 1787. 10. The "Holy Experiment" was tried out by William Bradford, William Brewster, Jacob Leisler, Peter Minuit, William Penn. 11. To help the poor debtors of England James Oglethorpe founded the colony of Conn., Del., Ga., Md., N. J., S. C., Va. 12. Indentured servants and slaves were held mostly in Del., Ga., Pa., S. C., Va. 13. The first settlement of the Catholics was in Conn., Del., Md., Mass., N. C., N. H., N. J., R. I. 14. The first Pilgrims landed in 1607, 1619, 1620, 1643, 1683, 1776, 1789. 15. The first college founded was Brown, Harvard, Pennsylvania, Princeton, William and Mary, Yale. 16. The most outspoken friend of the colonists in the English Parliament was Dinividdie, Montcalm, Pitt, Towns hend, Wolje. 17. As Royal Governor of New England Sir Edmond Andros represented the Common People, the Continental Congress, the English Crown, the Proprietors. 18. Thomas Hooker led emigrants from Mass, to found Conn., Del., Ga., Md., N. J., N. Y. 19. Nathaniel Bacon led a Rebellion against the tyranny of Governor Andros, Argall, Berkeley, Dale, Minuit, Winthrop, Yeardley. 20. The colony of Ga. was once invaded by the Dutch, French, Irish, Spanish, Swedes. 21. The New Sweden of 1638 later became known as the colony of Conn., Del, N. H., Pa., R. I. 22. The Mason and Dixon Line was established between Pa. and Del., Md., N. J., N. Y., Ohio. 23. The colony of South Carolina was noted for the very early settlement of the Baptists, Catholics, Huguenots, Lutherans, Methodists in the founding of Charleston. Scales and Tests 65 24. The territorial claims of the English in America were established by the treaty of Aix-la-Chapelle, America, Paris, Ryswick, Utrecht. 25. The Germans settled mostly in Md., N. ]., N. Y., Pa., R. I., S. C., Va. 26. Local business affairs in the New England settlements were generally managed by the governor, king, mayor, minister, town-meeting. SCORE SHEET FOR TEST E No. of Each Exercise The Correct Answer Approximate Value Sigma Value 1 Massasoit 2 2.04 2 Indians 2 2.05 3 1607 2 2.05 4 Pennsylvania 2 2.07 5 Puritans 2 2.12 6 1600 2 2.13 7 Rhode Island 2 2.16 8 Virginia 2 2.16 9 1619 2 2.26 10 William Penn 2 2.29 11 Georgia 2 2.29 12 Virginia 2 2.31 13 Maryland 2 2.34 14 1620 2 2.37 15 Harvard 2 2.37 16 Pitt 2 2.38 17 English Crown 2 2.38 18 Connecticut 2 2.38 19 Berkeley 2 2.39 20 Spanish 2 2.42 21 Delaware 2 2.45 22 Maryland 2 2.46 23 Huguenots 2 2.47 24 Paris 2 2.50 25 Pennsylvania 2 2.51 26 Town-meeting 2 2.53 66 Standardized Exercises in History TEST F 1. Among the many hardships of the times were the unexpected attacks by Arabian, Chinese, Cossack, Indian, Negro warriors. 2. The "Mayflower Compact" was drawn up by the Baptists, Huguenots, Methodists, Pilgrims, Quakers. 3. By the time the "Mayflower" had arrived at Plymouth Rock it had crossed the Antarctic, Arctic, Atlantic, Indian, Pacific ocean. 4. Under the "Treaty Elm" William Penn made a lasting peace with the Canadians, Huguenots, Indians, Negroes, Puritans. 5. The most commonly spoken language of early New England was Dutch, English, French, Scandinavian, Spanish. 6. By the opening of the Revolutionary War the number of colonies had increased to twelve, thirteen, fourteen, fifteen, sixteen. 7. New York was founded by the Dutch, English, French, Spanish, Swedes. 8. By the latter part of the 18th century the form of government most desired by the colonists was aristocracy, autocracy, democracy, monarchy, oligarchy. 9. The Great Treaty was between the Indians and Governor Berkeley, John Endicottj Cotton Mather, William Penn, John Smith, Miles Standish. 10. The Charter Oak was a church, meeting house, painting, school, town hall, tree. 11. The political control of the Dutch in America came to an end when Governor Stuyvesant surrendered to the English, French, Germans, Spaniards, Swedes. 12. The light at night in these early homes came from candles, electricity, gas, kerosene, phosphorus. 13. The most important of the Four Inter-Colonial Wars to the colonists was Queen Anne's, King George's, French and Indian, King William's. 14. The main export crop of early Va. was corn, cotton, indigo, rice, tobacco, wheat. 15. The Quakers founded the colony of Ga., Mass., N. H., Pa., S. C. 16. George Washington was fitted for leadership in the Revolutionary War through his experiences in the Colonial War of Queen Anne, King George, French and Indian, King William. 17. In emigrating the colonists generally went east, north, south, west. 18. The "Plymouth Rock" is in Conn., Del, Md., Mass., N. J., N. Y., Pa., R. L, S. C. 19. The last of the thirteen colonies founded was Conn., Del, Ga., Md., Mass., N. C., N. Y., Pa., S. C., R. L, Va. 20. The last colony was founded in the 15th, 16th, 17th, 18th, 19th century. 21. William Claiborne led a Rebellion against Lord Baltimore, Lord Berkeley, Lord Brook, Lord Delaware, Lord Say. Scales and Tests 67 22. "Braddock's Defeat" occurred in the first, second, third, fourth, Colo- nial War. 23. The gap in location, age, and ideals between the Northern and South- ern colonies was bridged over by the founding of Del., Md., N. J., N. Y., Pa. 24. The leadership of John Winthrop was noted in the founding of Baltimore, Boston, Charleston, Philadelphia, Providence. 25. The strict Puritan Church of early New England gradually gave way to what has since become known as the Baptist, Congregationalist, Lutheran, Methodist, Spiritualist church. 26. The Proprietary form of government was exercised in Mass., N. Y., Pa., R. I., S. C., Va. 27. The right of woman suffrage was granted for a time in Conn., Del., Mass., N. J., Va. 28. John Berkley and George Cartaret once owned Conn., Del., Ga., Md., Mass., N. H., N. J. 29. The Grand Model was drawn up by Cecil Calvert, King Charles, John Locke, John Smith. 30. The colony of Maine was once under the proprietorship of John Davenport, Sir Ferdinando Gorges, Anne Hutchinson, William Kieft, Roger Williams. 31. The welfare of the New York colony was much helped by the very early settlement of the Huguenots at Albany, New Burg, Ne