os cs /. 7!/3: BISON '/Y,8 'l:Pe...--- Job Analysis of the Entry Level Firefighter Position W SUNY AT BUFFALO United States THE LIBRAR!ESCMI Sen1ce Commilslon DEPOSITORY COPY Bureau of Polictes and Standards JOB ANALYSIS OF THE ENTRY LEVEL FIREFIGHTER POSITION Davild A. Bownas and Robert ~1. Heckman U. S. Civil Service Commission Contract # CS-1642 Dr. Charles H. Anderson, COR Personnel Decisions, Inc. 821 Marquette Avenue Foshay Tower Minneapolis, Minnesota 55402 612 339 0927 3 December 1976 This project was supported under contract number CS-1642 by the U. S. Civil Service Commission. Points of view or opinions stated in this document are those of the authors, and do not necessarily represent the official positions or policies of the U. S. Civil Service Commission. Acknowledgments In the course of performinp this job analysis, we have had the pleasure of working with well over 500 people in 100 departments across the country. We asked most of these people to perform some rather unexciting tasks involving long periods of concentration, and we frequently asked them to consider their jobs in ways that were relatively new and therefore uncomfortable for them. The response to these requests was uniformly one of enthusiastic cooperation and active involvement in our goals and problems. We heartily thank al 1 of the anonymous contributors to our project, for they are the true experts who have provided what we know about firefighters and their work. We can only hope that our interpretatioh and recording of the information they have offered constitutes a sufficiently accurate picture of their own expertise. While it is impossible to thank individually all the people who participated in our project, we cannot print this report without mentioning a group of men who 1iterally made the study possible. We worked with each of our departments through a liaison officer who voluntarily managed the flow of information, obtained workshop participants, scheduled alI department activities, and frequently performed a 1 ion 1 s share of the actual rating himself. We would, then, 1ike to thank the following people for the time and effort they contributed in the hope of advancing fire service selection. Albuquerque NM -Chief Arthur Martinez Alexandria LA_-Chief Charles Carruth Athens GA -Chief Thomas Eberhart Auburn NY-Chief William Maywalt Bakersfield CA -Chief C. C. Haggard Baton Rouge LA-Chief Miller Beaumont TX-Chief Chriswell Assistant Chief C. A. Shelton Bellingham WA-Chief J. Baker Billings MT-Chief Wallace Frickle Boise 10-Chief David Perry Boston MA-Commissioner George Paul Deputy Chief John Harrison Bozeman MT -Chief Bud Simpson Bremerton WA'-Chief Mex Meiqs Canton OH-Chief Thomas Kilkul Jan Assistant Chief Walter Clark Carlsbad NM-Chief C. C. Patton Cedar Rapids lA -r.hief Edsel McMickle Charlotte NC -Chief John Lee Chesapeake VA -Chief Robert Baqley Chula Vista CA-Chief William Smithey Cincinnati OH -Chief Burt Luqannani i i. Clifton NJ -Chief Stephen Lend! Colorado. Springs CO -Chief R. W. Nice Columbia SC -Chief Bert Dickert Coral Gables FL -Chief Borartfield Corvallis OR-Chief Walter Pflughaupt Danville VA-Personnel Director Dan Smiley Davenport lA-Chief Howard Goettsch Decatur GA -Chief H. B. Paden Eugene OR -Chief Everett Hal 1 Everett WA -Chief Reina Watson Assistant Chief Larry Dressel Fairfield CA-Chief Richard Silves Assistant Chief Gary Eberly Fremont NE -Chief Howard Schneider Fullerton CA-Chief Ray Eiler Operations Chief Houser Great Falls MT-Chief Kenneth Lindstrand Harrisburg PA -Chief Chester Henry Hialeah FL-Chief \olilliam Lynch Inglewood CA-Chief Hil 1 Jersey City NJ -Acting Chief Duncan Johnstown PA -Chief Charles Krumenacker Lawrence MA-Chief Richard Lacey Littleton CO-Chief Johnson Captain James Etzel Long Beach CA-Chief Virgil Jones Los Angeles CA -Chief Kenneth Long Assistant Chief George Donne! Louisvi lie KY-Chief Jack Cummins Lowell MA-Chief Paul Beauregard. Lubbock TX -Chief Haske] 1 Keeton Assistant Chief Foster Lynwood CA-Assistant Chief R. A. Lathrope Marion IN-Chief Fred Shinault (Retired) Chief Richard Stevenson Marshalltown lA-Chief Leonard LaPour Training Director James Simmons Melrose MA-Chief Francis O'Brian Menomonee Falls WI -Chief John Fulcher Miami FL -Chief Don Hickman Middletown OH-Chief Victor Auvil Millbrae CA-Chief Robert Kohler Milwaukee WI -Chief William Stamm Montgomery County MD -Chief David Gratz Lieutenant Joho Best Nashville TN-Chief Ackerman Natick MA-Chief Richard Fahey New Haven CT -Chief Francis Sweeney Deputy Chief Walter Brocar Norfolk VA-Chief K. W. Sykes iii. Oakland CA-Chief Steve Meniette (Retired)Chief William MooreAssistant Chief Walter PiersonOdessa TX-Chief W. J. ChildressOgden UT -Chief Charles HansenAdministrative Assistant Don Hi !tonOklahoma City OK-Chief Byron HollanderDeputy Chief Carl HolmesOmaha NE -Chi.ef Vernon VanScoyOrange TX -Chief M. J. Girl inahousePhiladelphia PA-Deputy Chief Wil 1iam RichmondPittsburgh PA-Chief Thomas KennellyPontiac Ml -Chief Albert RaynerPortland OR -Chief Gordon MorterudPrescott AZ-Chief Jack HamiltonProvidence Rl -Chief M. F. MoisePueblo CO -Chief James LovettPullman WA-Chief Patrick WilkinsRenton WA-Cbief George WilliamRichmond CA-Chief Raymond MartinRiverside CA-Chief Fred WoodwardSt. Louis MO -Chief Charles KampradDeputy Chief Harry HullSalem OR-Chief Wesley BakerSal ina KS -Chief BossSalinas CA-Chief Paul MehringerSalt Lake City UT -Chief Leon DeKorverSan Antonio TX -Chief I. 0. MartinezSan Diego CA -Chief Dee RogersSan Francisco CA-·thief Keith CaldenAdministrative Chief CasperSan Jose CA -Chief John GerhardtDeputy Chief Earl ThompsonSanta Ana CA-Assistant Chief Ray BachpelleSanta Clara CA-Chief D. P. ViscontiSanta Cruz CA -Chief Charles SchneiderSavannah GA -Chief John SchroderSeattle WA-Assistant Chief Cleo HathawayScottsdale AZ -Chief Louis WitzemanSioux City lA -Chief Robert ArnoldTraining Chief Sidney SkidmoreSpokane WA -Chief A. L. O'ConnorSpringfield OH -Chief Frank TrempeSpringfield OR-Chief Virgil NaveStillwater OK-Chief Frank SmithSutter County CA-Fire Marshall Jack BeckerSyracuse NY -Chief Thomas Hanlon.Deputy Chief Hess iv. Temple TX -Chief Rol ins Bearor Fire Marshall James Conely Toledo OH -Chief Eulan Tucker (Retired) Assistant Chief R. G. Seelman Topeka KS-Chief Ben Neill Tucson AZ-Chief L. F. Peterson Deputy Chief W. G. Dearing Utica NY -Chief Gerald Darby Vancouver WA -Chief James Brown Wauwatosa WI -Chief Byard Sheldon West Orange NJ -Chief Frank Capron Wichita KS -Chief F. E, Hobbs Williamsport PA-Chief Vincent Mancine We also wish to thank Chief Clarence Nimmerfroh of Minneapolis and Chief Steve Conroy of St. Paul for making their facilities available to us for pretesting parts of our surveys. In addition, several people on the Personnel Decisions staff made valuable contributions to the project, through their hard work, long traveling hours, and creative insights. First, Ste~e Motowidlo deserves primary credit for most of the favorable aspects of the-study. In originally designing the research plan we followed, Dr. Motowidlo outlined the most thorough and comprehensive strategy for construct validation we have seen to date. Any flaws in the research reported here are in the execution, not the desian. Several other people developed materials, collected data, and contributed ideas to our study, including Bill Hanton, Rob Silzer, and Jack Rossmann who were extremely active during the early stages 6f research. The contributions of Rod Rosse, our director of research design and statistics, enabled us to perform statistical analyses of overwhelming masses of data accurately and efficiently. Linda Martinson performed valuable administrative services throughout much of the project. Suggestions by Marv Dunnette, Wally Borman, and Leaetta Hough vastly improved the quality of the product, as did the helpful advice of our project manager, Dr. Charles Anderson. Finally, the seemingly tireless efforts of Jana Odegaard and Mary DeGree in preparing rating materials, scheduling workshops, makinq _traveling arrangements, and most of all, repeatedlytyping sections of this report with almost daily promises that now it was ready to go to press, truly made it possible to perform and describe the research reported here. To alI these people, we give our most sincere thanks. v. TABLE OF CONTENTS "' I. Executive Summary Objectives Proceoure Results 2 II. Introduction 3 Project Background 3 Literature Review 3 Job Analysis 6 Job Performance Measures . 20 Selection Procepures . 30 Selection Interview ... . 31 Physical Ability T.esting . 31 Paper and Pencil Tests . 36 Ill. Organization of the Job Analysis Report ........43 IV. Sample . .............. . 44 V. Identifying Firefighter Tasks. 48 Definition of the Target Job . . . . 48 Development of a Preliminary Task Checklist 48 Site Visits to 18 Departments . . . . . . . : 49 VI. Procedures and Results of Task Analysis Check! ist Administration ...... . . 51 VI I. Clustering Tasks · · · · .. 56 Rationale for Clustering ..· 56 Simi.larity Rating Task 56 Reliability and Cluster Analyses of Similarity Ratings ...... . 57 VI II. Regional Workshops to Generate Information about Task Clusters· .............. 63 vi. IX. Nationwide Survey Questionnaires to Generate lnformat ion about Task Clusters .... 70 [• Rationale and Questionnaire Development 70 Questionnaire Sample ..... 71 Dimension Importance Ratings 71 Task Cluster Difficulty Ratings 77 Initial Ability Requirement Ratings 79 Attribute-Cluster Importance Ratings 80 Critical Incident Scaling and Retranslation 86 X. Implications of Job Analysis Results for Developing Written Testing Procedures 90 vie ighted Examination P1an 90 Measures of Attributes 95 Implications of Tests for EEO Laws 35 Adapting Procedures to Local Practices 97 XI. Use of Job Analysis Results of Other Jurisdictions 98 Background . . . . . . . 98 Developing Job Similarity Assessment Technique . . . . . . . . . . 99 Procedure for Assessing Department Similarity 100 XI I. Summary and Conclusions .........•.... 104 Reference Notes . . . . . . . . . . . . . . . . . . 105 References . . . . . . . . . . . . . . . . . . . . . 106 Appendices . . . . . . . . . . . . . . . . . . . . . 111 vii. TABLES 1. Percent and Frequency of Task Performance for 7 Selected Tasks from A Study of the Fireman's Oc~upation 2. Job Performance Dimensions Developed by Personnel 22 Decisions in Minneapolis, Grand Rapids, and St. Paul Firefighter Validation Projects 3. Behavioral Classifications of Atlanta Commission 25 Job Performance Dimensions 4. Abi 1ity Tests Recommended by AIR for New York City 35 Firefighter Screening 5. Major Project Phases and Participating Jurisdictions 45 6. Regional lntercorrelations for Task Importance and 54 Time Spent Ratings 7. Size lntercorrelations for Task Importance and Time 55 Spent Ratings 8. Hoyt Reliabi 1ities for Ratings of Expected Performance 58 Similarity Between Tasks 9. Factor Loading Matrix for Five Factors and 16 Task 66 Clusters 10. Factor Loading Matrix for Three Factors and 15 67 Physical Abi 1ities 11. Factor Loading Matrix for Four Factors and 20 68 Perceptual and Cognitive Abi 1ities 12. Factor Loading Matrix for Six Factors and 26 69 Personal Attributes 13. Task Dimension Importance Means and Standard 73 Deviations and Percent of Departments Giving Non-zero Ratings 14. Regional and Size lntercorrelations for Task 74 Dimension Importance Ratings 15. Mean Ratings of Importance of 20 Attributes for 81 Performing Job Dimensions at an Adequate Level viii. 16. Mean Ratings of Importance of 20 Attributes for 82 Performing Job Dimensions at a Superior Level 17. Re1 iabi lity of Ability Importance Ratings Computed 83across 20 Abi 1ities 18. Distribution of Completed Incident Retrans1ation 88Rating Forms by Region and Department Size 19. Weighting Plan for 20 Attributes and 16 Performance 93 Dimensions, Based on Ratings of Attribute Importance for Adequate Performance 20. Weighting Plan for 20 Attributes and 16 Performance 94Dimensions Based on Ratings of Attribute Importancefor Superior Performance ix. FIGURES l. Performance scales identified for firefighters in Montgomery, Alabama. 2. Important PAO. dimensions identified by Atlanta 12 Regional Commission (1974). 3. Most important KSAP 1 s as rated by Des Moines firefighters 15 in Ruedebusch et a 1. ( 1975). 4. Duties identfied for Fort ~!orth firefiqhers by Dallas 16 Regional Commission. 5. Job elements identified for Fort Worth firefighters 17 by Dallas Regional Commission. 6. St. Paul firefiqhter performance rating scale 23 (Heckman, 19l3b). 7. Job performance dimensions tapped by interviews, 32 Selection Consulting Center. 8. Map of six geographical regions included in 46 nationwide entry-level firefighter study. 9. Example of clustering sequence for ten rescue tasks. 60 10. Task dimension factors. RS 11. Plot of ranked dimension importance means. 91 14. Plot of task cluster importance-time spent cross101 product means for most extreme and second most extreme departments for 16 clusters. 13. Recommended cross-product task cluster importance 102 cutoff scores for identifying departments similar to development sample. x. APPENDICES A. Departments Par~icipating 1n Nationwide Entry Level Firefighter Job Analysis B. Agenda and Check] ist used during Phase 1 Site ·Visits to 18 Fire Departments C. Phase II Firefighter Task Analysis Checklist D. Firefighter Task Time Spent and Importance Means and Standard Deviationsand Percentages of Respondents Indicating Tasks Performed on a Rei iefBasis On 1y E. Instructions and Materials for Rating Task Similarity and IdentifyingRequired Abilities in Phase I I I Workshops F. Mean Phase Ill Similarity Ratings for Pairs of Firefighter Tasks G. Preliminary List of 62 Knowledges, Skills, Abilities, and otherCharacteristics H. Conditions Affecting Task Performance Difficulty I. Definitions of 20 Knowledges, Skills, Abilities, and other Charact~ristics, and of 17 Firefighter Task Clusters Developed for Phase V Task AnalysisSurvey J. Cover Letter Accompanying Phase v Task Analysis Survey Materials K. Phase v Task Dimension Importance Questionnaire L. Phase v Task Difficulty Rating Materials M. Phase v In it ia 1 Ab i 1i ty Requirements Rating Materials N. Phase v Ab i 1 i ty Importance Rating Materia 1s 0. Phase v Critical Incident Retranslation Materials P. Tentative Behavioral Performance Scales Q. Recommended Tests for 20 Attributes ('. R. Procedures for Determining Applicability of Recommended FirefighterSelection Battery for Fire Departments I. EXECUTIVE SUMMARY Objectives. This paper reports the procedures and results of a nation wide job analysis of the entry-level firefighter position. The research described here is funded by the United States Civil Service Commission (USCSC), and is intended to further the goals of the Equal Employment Opportunity Coordinating Council (EEOCC). These goals include encourag ing the development and validation of usable selection procedures for high-volume public jobs, and the current study is supported by USCSC as a demonstration research project complying with EEOCC's intentions. Our study was intended to identify the major tasks performed by entrylevel firefighters across the United States. We sought to assess the difficulty of performing each such task, the conditions under which tasks are performed, the relative importance of various job duties, and the knowledges, skills, abilities, and other characteristics (KSA's) required to perform tasks effectively. Finally, we sought to specify standards for performing firefighter duties at an adequate and at a superior level and to develop an examination plan showing the relative weights to be given to each KSA in predicting job performance. Procedure. We included in our sample a total of 109 fire departments, geographically dispersed, of sizes ranging from fewer than 25 to nearly 3,000 uniformed personnel, and representing cities of from no more than 20,000 to more than 2.5 mil 1 ion population. Comparisons with available data showed that racial minorities were represented in the job analysis phase:s of our study in approximately the same proportions as those in which they occur in the fire service nationally. Our study began by identifying, through literature reviews, interviews, and observations, the tasks performed by firefighters in U.S. fire departments. A task analysis questionnaire was developed from these tasks and distributed to a nationally representative sample of fire departments to estimate the relative importance of each firefightertask. In the next stage, the tasks identified as most critical were categorized into homogeneous clusters, based on estimates of task performance similarity, and KSA's important for performing each task were suggested. In a series of workshops, fire service professionals gave us more information about KSA's and the task clusters, described the conditions under which the tasks in each cluster are performed, and wrote examples of inadequate, adequate, and superior performance for each cluster of tasks. Finally, fire personnel from a national sample of more than 50 departments rated the .importance and difficulty of performing various firefighter tasks and clusters, estimated the amount 2. of each KSA required of firefighter recruits at the tim~ of hire, and rated the importance of ~ach KSA for performing in each iask cluster at an adequate and at a superior level. Results. At every step in our project, we found excellent agreement among raters, among departments, among regions, and among department sizes for all rating tasks. Sixteen clusters comprising 109 important firefighter tasks were identified durinq our job analysis. These task clusters, or job performance areas, include: 1. Performing rescue operations 2. Performing salvaqe and overhaul 3. Performing ladder operations ~ Forcibly entering structures and enclosures 5. Applying ventilation procedures 6. Following standard safety procedures 7. Applyinq knowledge of fire characteristics 8. Performing hose evolutions and applying extinquishina aoents 9. Operating apparatus 10. Administering emergency care 11. Dealing with the public 12. Performing preplannina and fire prevention inspections 13. Participating in traininq and education 14. Reconditioning ·and maintaining equipment 15. Performing routine station duties 16. Getting along with peers Ve identified 20 knowledqes, skills, abilities, and other characteristics reliably rated by fire service experts as contributino to performance in these 16 job areas, recommended instruments for measuring each KSA, and derived the relative weights to be given to each such attribute in estimating future performance as a firefighter. Finally, we have developed procedures which departments can use to ascertain the similarity of their own firefighter positions to those on which our analysis was conducted. These procedures can be used to test th~ appropriateness of the selection procedures described in this report for selecting firefighters in any jurisdiction. I I . INTRODUCTION 3. Project Background Personnel Decisions, Inc., (PDf) began work July 1, 1975, on a project titled Nationwide Job Analysis of Entry-Level Firefighter Positions in Local Governments under Contract CS-1642 to the U. S. Civil Service Commission (USCSC). The project is being performed within the general framework bf the proposed Equal Employment Opportunity Coordinating Council (EEOCC) Uniform Guidelines on Employee Selection Procedures. EEOCC was established by the Equal Employment Opportunity Act of 1972 and is composed of the Secretary of Labor, the Chairman of the Equal Employment Opportunity Commission, the Attorney G~neral, the Chairman of the U. S. Civil Service Commission, and the Chairman of the U. S. Civi 1 Rights Commission, or their delegates. EEOCC proposed that cooperative projects be sponsored (1) to demonstrate how selec~ion procedures could be validated in state and local government agencies consistent with the Uniform Guidelines and (2) to explore the feasi bility of a multijurisdictional approach rather th~n independent efforts by state and local governments. EEOCC established a priority list of occupations for which such studies should be conducted, including the position of entry-level firefighter. The major objectives of this project are to conduct a selection oriented job analysis of the firefighter position in local governments, and fo develop model selection instruments for two major qua1ification areas of the firefighter position. Because previous work has rather thoroughly explored cognitive (i.e., mental abi l.ity) and physical selection proc~dures, these types of predictors are excluded from the scope of this study. This report summarizes accomplishments on the first objective--performing a selection oriented job analysis. Our final report, to be completed early in 1977, will describe complete results of the project, including the development and validation of firefighter selection instruments. Literature Review POl reviewed the published and unpublished firefighter job analysis, test development and validation, and performance evaluation literjtur~ to provide a foundation for the sJb~equent phases of the project. Although our review was not meant to be exhaustive, we feel it is repre~entative of the work currently being performed in the area. Surveys of Firefighter Activities Performed by Professional Fire Service Organizations The most frequently cited document describing act1v1t1es of and pro fessional standards for firefighters is Fire Fighter Professional Qualifications established by the Joint Council of National Fire Service Organizations and published by the National Fire Protection 4. Association (NFPA, 1974). A committee including fire chiefs, firefighters,racial minority firefighting pe~sonnel, training instructors, and fire service associations was charged with developing performance standards in terms of behavioral objectives. The standards are considered the minimum requirements for demonstrating competence in providing fire service at the level and in the role specifi~d. The standards for Fire Fighter Professional Qualifications cover both entrance requirements and the requirements for subsequent levels of progression, i.e. Firefighter 1, 11, and 111. Although the NFPA standards are lengthy and involved, they include the following general entrance requirements: 1. high school diploma or state recognized equivalent 2. at least 18 years of age 3. medical exam (the standards contain 28 pages of medical requirements) 4. character investigation and evaluation 5. pass minimum physical fitness requirements including: a. run 1.5 miles in 12 minutes b. 25 bent-knee sit-ups within 90 seconds c. 5 chin-ups d. 10 push-ups e. walk a 20-foot long, 3 to·4 inch beam carrying a roll of fire hose weighing at least 20 pounds f. I ift a 125-pound weight and carry Jt 100 feet without stopptng g. move a IS-pound weight from left to right (7 times in each direction) in a prescribed manner in less than 35 seconds. Perhaps more important than the ehtrance require~ents was the descriptionprovided in the standards of Firefighter 1 performance objectives. These behavioral objectives were listed under the following categories: general;forcible entry; protective breathing apparatus; first aid; ropes; salvage;fire hose, nozzles, and appliances; fire streams; ladders; ventilation; inspection; rescue; sprinklers; fire alarm and communications; safety;and fire behavior. Firefighter I I standards define more advanced objec tives and include water supplies, portable extinguishers, and overhaul categories. More stringent standards are specified for Firefighter I I I certificat.ion. Another monumental project by a professional fire service group is represented by the International Fire Service Tr~ining Association (IFSTA) publications. IFSTA is an educational all lance organized to develop training materials for the fire service. Fire Protection· Pub! ications at Oklahoma State University, Stillwater, Oklahoma, publishes al 1 s. IFSTA training materials and texts. Periodically, knowledgeable fire officials from across the country meet at Stillwater to discuss, update, and approve the IFSTA materials. Although IFSTA has many publications, the most relevant for our study of the firefighter position are: Pub 1i cation Number Title 1OJ For c i b1e En t ry 102 Fire Service Ladder Practices 103 Fire Hose Practices 104 Salvage and Overhaul Practices 105 F re Stream Practices 106 F re Apparatus Practices 107 F re Ventilation Practices 108 F re Rescue Practices 109 F re Service First Aid Practices 110 F re Inspection Practices These publications, compr1s1ng more than 1600 pages of instructional information, provided POl with a rich source of task and job information. Since most states have adopted the IFSTA publications as official training materials, we were confident that these documents would prove an extremely valuable source of job information. Another noteworthy project was completed by the Illinois Fire Protection Personnel Standards and Education Commission. The Commission developed a Certified Firefighter training course consisting of 200 hours of lecture and drills in 23 subject areas, implementing the program in January, 1973. The Commission took a systems approach for defining measurable objectives and developing course materials which enable students to achieve these objectives. The course outline and subject matter areas developed by the Commission offered support for several ideas we had developed for classifying firefighter tasks, and their comments and suggestions about factors contributing to firefighter per formance were influential in subsequent phases of our study. Although the purpose of this literature review is not to identify or evaluate work of alI local, state, or national fire service organizations, PDI would be remiss if we did not mention the extensive task analysis completed by the Oklahoma City Fire Department. Oklahoma City conducted a functional job analysis for each of several positions in the Department. They developed lists of specific tasks, and they have plans for rating each task on worker function scales for Things, Data, and People. Oklahoma City•s approach also involves analyzing firefighter positions using three general educational development scales and a specific vocational preparation scale. POl drew heavily from their t~sk lists during early job analysis phases of our project. 6. Job Analysis The most comprehensive analysis of tasks performed by firefighters is documented in the historic Study of the Fireman's Occupation (Allen, Bodner, Lano, and Meyer, 1973). This report serves as the foundation of the California Fire Service Certification Program. Discussions between the California Fire Chiefs' Association and the California State Department of Education's Bureau of Industrial Education led to a decision to conduct a comprehensive analysis of the fireman's occupa tion in California. The Division of Vocational Education, University of California, contracted to perform an occupational task analysis of alI ranks within the fire service. Based on the study's results and on reviews by various advisory committees, a suggested curriculum was developed for pre-employment, pre~promotional, and post-promotional training for each of the ranks. A three-man subcommittee of the Fire Training Program's State Advisory Committee, the State Fire Training Staff, and 'th~ research staff worked together to finalize the data col l~ction questionnaire. Unfortunately, the pre! iminary job analysis steps leading to the questionnaire are not described in the report. Ninety departments participated in the study including 41 small depart'ments (under 50 men), 30 medium (50 to 150) and 19 large (over 150), providing a total of l I ,660 fire service personnel from nine ranks. In part I of the study a 377-item task inventory was administered to Captains, Engineers, Firemen, Fire Prevention Inspectors, Alarm Operators, and Haster Hechanics. In Part II, a 71-item supervisory and administrative t~sk inventory was administered to higher level personnel. The first part generated the following information for each task: (1) the percent of individuals by rank and by department size performing each task; (2) frequency of task per formance; (3) technical knowledge level required, using Bloom's (1965)classification of educational objectives; (4) conditions under whichvarious manipulative tasks are performed; and (5) the degree of training required for each task. The results of this monumental project are a series of tables depictingstatistical results, post-employment training needs for each task, andStatewide Advisory Committee training recommendations. The tasks are generally well defined; for example, Inspect and Maintain SalvageCovers is described as "includes inspecting, cleaning and testing waterrepel lent characteristics of canvas and other types of salvage covers;refolding and patching salvage covers; and maintaining records.'' Tableprovides the percent of Firemen performing each task, and the frequencyof performance by size of department, for four tasks. ]. TABLE 1 PERCENT AND FREQUENCY OF TASK PERFORMANCE FOR SELECTED TASKS FROM A STUDY OF THE FIREMAN'S OCCUPATION Percent Performing Frequency s M L s M L Maintain Fire Station Grounds 79 93 96 H H H Inspect and Maintain Salvage Covers 84 81 91 M M M Operate Prying Tools 89 92 90 H H H Address the Public 36 17 14 H M H *Under Percent Performing S, M, and L refer to Small, Medium, and Large. H, M, and L refer to High, Medium, and Low under Frequency. 8. I~ their conclusions, the researchers make no mention of similarities and differences between smal 1, medium, and large departments. Presumably they felt that since departments of all sizes participated, the results and recommended training would be broadly applicable. Furthermore, although the report prov~des no rationale for grouping related tasks together, it does 1ist groups of tasks which seem related in a series of 51 tables. For example, one table labeled 11 Buildings and Grounds11 contains eight tasks such as maintain fire station, oqtain house supplies, make minor station repairs, and maintain fire station. grounds. The table headings could thus be viewed as broad tasks while the table elements could be termed sub-tasks. The Selection Consulting Center (SCC), California State Personnel Board,relied heavily on A Study of the Fireman's Occupation in its projectThe Validation of Entry-Level Firefighter Examinations in the States ofCalifornia and Nevada (1973). SCC staff interviewed personnel in 38 of73 participating jurisdicti9ns and content analyzed the interview datato identify six performance dimensions: (1) Mechanical and TradesKnowledge; (2) Physical Abilities; (3) Personal Attributes; (4) Communication Ski 1ls; (5) Problem Solving; and (6) Mechanical/Spatial Aptitude. Based on the interviews and on the Study of the Fireman's Occupation, sec developed a questionnaire to obtain importance ratings for tasksand for abi 1i ties, personal characteristics, attributes, and other potentialcontributors to performance. Each of the six dimensions identified in theinterviews was represented by at least four statements in the questionnaire. These inventories were completed by a total of 284 respondentsfrom 62 jurisdictions. sec averaged the importance ratings for all statements representing adimension to obtain a dimension importance rating. All dimensions wererated as having at least moderate importance (greater than 3 on a 5-point scale). Physical Abilities was rated as most important followed byPersonal Attributes, Mechanical and Trades Knowledge, Problem Solving,Mechanical and Spatial Aptitude and Communication Skills, respectively.Respondents from smal 1 (less than 50 personnel), medium (50 to 150),and large (more than 150) departments closely agreed on the importanceof these dimensions. The largest difference was between importanceratings of smal 1 and medium departments on Physical Abi 1ities (4.1 vs. 4.4on a S-point scale). sec used these job analysis results to develop predictors and to formulate performanc~ evaluation scales. Test publishers were encouraged to writeinstruments which might predict the cognitive and knowledge dimensions. 9. Several researchers have utilized the Position Analysis Questionnaire (PAQ) (McCormick, Jeanneret and Mecham, 1969) to study the firefighter posrtron. The PAQ is a structured job-analysis questionnaire consisting of 194 job elements. The elements are termed "worker oriented11 since they imply human behaviors involved in jobs, as contrasted with job-or task-oriented statements which deal more with technological processes of the jobs themselves. The PAQ can be computer analyzed to derive 27 fairly specific job dimension scores and scores on five overall job dimension factors. Cecil and Cecil (Note l) used the PAQ in the ~1ontgomery, Alabama, Fire Department as part of an empirical val idation'study. A number of teams, formed by pairing a fireman with a job analyst, completed the PAQ. Ceci 1 and Cecil report that the firefighter position in Montgomery scored two or more standard deviations above the mean on the following job dimensions: being aware of environmental conditions; performing activities regarding general body movements; and being in a hazardous or unpleasant environment. Being physically active/related env~ronmental conditions was the highest overall dimension score for the firefighter pos1t1on. Cecil and Ceci 1 concluded from the PAQ analysis that four general factors are of major significance in the firefighter 1 s job. First, the firefighter must be physically able and must possess the stamina to engage in strenuous activities over a prolonged period in unpleasant and often hazardous conditions. Second, the firefighter must have a highly developed sense of balance and orientation, and good eye and hand motor coordination. Third, he must be constantly aware of changing conditions in the work environment and be capable of adjusting his activities immediately and appropriately. Finally, the firefighter must be willing to exchange information in the interest of learning and must get along well with his co-workers, supervisors, and the public. Ceci 1 and Cecil also conducted a "traditional" job analysis. They examined the recruit training program, its curriculum, and the testing procedures used in training. They also extensively questioned firefighters in every grade and asked them to demonstrate some of the particular skills and operations expected of people in each grade. The researchers then generated a list of specific skills, traits, attributes, and areas of knowledge, and constructed 23 performance rating scales using a Likert format with behavioral descriptions anchoring the scales (see Figure 1). All 23 scales.were rated on importance by fire depar-tment personnel. The knowledge scale of Forcible Entry, the performance scales for Rescue O~erations and Breathing Apparatus Operation, and all four attitude scales (Job Interest, Adaptation to Change, Dependability, and Getting Alonq with Others) were rated as being highly important. Cecil and Cecil concluded that these job analysis data gave support to mechanical aptitude as the single most important measurable ability for all ranks of firemen. They also indicated that personality factors would 10. Scales Organization and Chain of Command Rules and Regulations Communications and Signals Salvage Operations Generators and Lights Forcible Entry Ventilation Basic Fire Chemistry Fire Apparatus Lad·der Practices Basic Hydrau I i cs First Aid Practices Rescue Operations Sprinklers and Standpipes Hose Practice Rope Practice Breathing Apparatus Operation Aerial Ladder Practice Radiological Monitoring Job Interest Adaptions to Changing Situations Dependability Getting Along with Others Figure I. Performance scales identified for firefighters in Montgomery, Alabama. (Cecil & Cecil, Note I) be a fruitful area for investigation due to the close living arrangements, the need for discipline and coordinated action, and the need to tolerate stress. Cecil and Cecil discounted physical attributes since they felt that standards had been established over years of practice and were routinely used in hiring. The American Institutes for Research (AIR) also used the PAQ in a study of New York City's firefighter position (Brumback et al, 1974). In an eariJcr study, Marquardt and McCormick (1972) obtained ratlngs of the relevance of 76 human attributes for each of the 187 job elements in Form B of the PAQ. Brumback et al. had the PAQ completed for the New York firefighter position, identified all job elements rated above 2.5-on a 5-point scale, and referred to the earlier study to identify attributes relevant to the important job elements. The a·uthors ccmcluded that five constructs measureable with written tests were required by .the important job elements: understanding information, reasoning and problem solving, dealing with others, understanding mechanical devices, and spatial orientation/visual memory. Brumback et al. developed a matrix showing the contYibutions of these five constructs to the job elements important for firefighting, and developed a testing scheme which weighted each construct by its relevance to the elements and by the elements' i'mportance for job performance. Ultimately, 35 percent of the items in their recommended test battery were to tap understanding job information, 25 percent reasoning and problem solving, 10 percent mechanical comprehension, ~nd 10 p~rcent spatial and vis ua 1 a b i 1 i t i e s . Another project also made use of the PAQ. The report was prepared by the Technical Assistance Division of the Atlanta Regional Commission (1974) and financed in part by funds provided by the U. S. Civi 1 Service Commission under the Intergovernmental Personnel Act arid through the Integrated Grant Administration Program. The PAQ was administered to at least three officers in each of five jurisdictions. AI I the officers had been firefighters at one time and vJere currently supervising firefighters. Thirteen of the 27 specific job dimensions of the PAQ were judged to be important by the researchers based on the PAQ Profile scores. These 13 dimensions are presented in Figure 2. The Atlanta researchers also collecte·d critical incidents ·illustrating various levels of firefighter performance from members of the participating fire departments. Each incident was typed onto a 3 x 5 index card, and using the dimensions identified in the PAQ analysis as a conceptual framework, the incidents were sorted into categories. Although the researchers found they were unable to sort incidents into the original dimensions, they did identify a meaningful set of 15 cate9ories that reflected the PAQ information 11 to a moderate degree [p. 58]. 11 The researchers then grouped the incident category titles into four broader behavioral classifications as shown below: 12. Interpreting what is seen or heard Using data originating with people Watching things from a distance Evaluating information from things Being aware of environmental conditions Being aware of body movement and position Using hands and arms to control/modify Using feet/hands to operate equipment/vehicles Performing activities requiring general bodily movement Exchanging job related information Dealing with the public Being in a hazardous/unpleasant environment Being alert to details/changing conditions Figure 2. Important PAQ dimensions identified by Atlanta Regional Commission (1974). I 3. I. Job Knowledge A. Specialized job knowledge B. Interest in learning C. Helping others learn II. Interpersona 1 Re 1at ions D. Relationships with peers E. Relationships with the public II I. Approach to Work F. Willingness and eagerness to do his part of the work at the station G. Wi 11 ingness and eagerness to do his part of the work at the fire H. Respect for property and equipment I. Professional ism IV. Job Performance J. Remains calm in personally demanding situations K. Ability to perform physically demanding tasks L. Abi 1ity to follow orders M. Responsibility and safety of others N. Abi 1ity to make correct decisions 0. Abi I ity to work effectively in hazardous/unpleasant environment Ruedebusch, Vaaler, Keck, Powers, Mendenhall, Bayreder, and Black (1975) conducted a job analysis and test validation project in Des Moines, Iowa. Ruedebusch et al. first conducted job analysis interviews with 34 firefighters and I 1 officers to identify the major tasks performed by the Des Moines sample. Next, a 175-item task questionnaire was developed and was completed by 108 firefighters and engineers. Questionnaire respondents indicated for each task whether they performed the task at al 1; whether performance was routine, incidental, or occasional; the relative amount of time spent performing the task; and the task 1 s difficulty in terms of interpersonal, physical, and mental demands. Using some unspecified method, Ruedebusch et al. col lapsed the 175 tasks into five major groupings: (I) investigating complaints and making routine inspections; (2) studying and participating in daily drills and discussions; (3) responding to fires; (4) inspecting, servicing, cleaning, and maintaining equipment and apparatus; and (5) 'performing routine duties in the station between alarms. In a series of workshop sessions, Ruedebusch et al. asked 45 firefighters first to brainstorm the knowledges, skil Is, abi 1ities, and personal characteristics (KSAP 1 s) required to perform each of the five major task 14. groupings, then to indicate the extent to which each KSAP was trained after entry, and finally to identify the four most crucial KSAP 1 s for each task grouping. The ten most important KSAP 1 s ov~r all five task groups are listed in Figure 3. Ruedebusch et al. eventual Jy used their job analysis information to develop selection meas~res and to define performance dimen~ions. In a study similar to that of Ruedebusch et al., the qal las Regional Commission (Note 2) performed a job analysis and developed an examina tion plan for the Fort Worth, Texas, fire department. The researchers interviewed post-probationary firefighters with less than six years experience, observed firefighters on the -job, reviewed existing job descriptions, and.held discussions with fire ~upervi~ors, eventually classifying 70 tasks into eight major_firefighter duties {see Figure 4). For each of these duties, the researchers identified the major knowledges, skills, abilities, physical activities, tolerance for environmental conditions, temperaments, interests, general educational and specific vocational demands or requirements imposed by fire service work. Firefighters and supervisors generated a 1ist of 39 job elements or abil ities··and other factors required for job performance, and rated, foreach element, the element 1 s importahce'for ad6quate and for superior performance, the anticipated performance decrement if applicants are notscreened for the element, and the practicality of requiring that appl~cants be qualified in the element. These values were combined in accordance with Primoff 1 s (1975) Job Element procedure to identify the elements contributing most to successful performance, and more specific subelements (specific applications of the broader abi 1ities to firefighting situations) were generated to define the elements. After these steps,the researchers had identified 13 elements comprising 101 sub-elements The elements are I isted in Figure 5. These elements were matched againstthe eight duty categories identified in the earlier job analysis toensure that alI were related to one or more duties. An examination plan was developed to tap the 13 job elements. Becauseno strong rationale existed for developing differential weights for the13 elements, and because such differential weighting schemes are difficult to justify when based on relatively imprecise and unreliable estimatesof element importance, the elements were weighted equally in the plan.To maximize the relevance of test results to job performance, the researchers recommended using simulation exercises and work sample tests to measure most elements although written tests were suggested forreading, m~th and some sub-elements of mechanicgl ability. 15. Physical strength and agility Dependable as a member of the team Manual dexterity Carry heavy equipment/persons Ability to communicate with public in all situations Mechanical underst~nding Diplomacy, ma1ntaining good public relations Adapt, learn, and apply learning to specific situations Stamina, endurance Ability to compromise and get alonq with others Figure 3. Most ..Important KSAP's as rated by Des Moines firefighters in Ruedebusch et al. (1975). 16. Clean station house and station qrounds (daily) as part of maintenance standards of hygiene and order within station Maintains truck and firefiqhtinq equipment and tools to insure proper functioning Extinguishes fires Observes fire scene for possible arson clues Overhauls and salvages at fire scene Conducts fire inspections Administers first aid in emergency situations Attends wreck scenes Figure 4. Duties identified for Fort Worth fireifqhters by DallasReqional Commission (Note 2) 17. Maintain performance under difficult or stressful conditions Ability to work with minimum supervision Confidence in self and own abilities Mathematical abilities Reading ability Mechanical abi 1ity Ability to get along with others Judgment Willingness to take orders Willingness to learn Teamwork Physical abi 1ity Ability to remember details Figure 5. Job elements identified for Fort Worth firefighters by Dallas Regional Commission (Note 2) 18. In summary, the literature we rev-iewed discloses little consistency in job analysis approaches. Some reports provide only a brief description of job analysis methodology, while others go into great detail. The traditional interview approach provides maximum flexibility but limits generalization across jurisdictions because of its sub_jectivity. Job analysis interviews produce a variety of outputs including general narrative descriptions, specific 1ists of tasks and duties, 1ists of personal attributes required to perform the firefighter job, and descriptions of environmental conditions encountered and equipment employed" The questionnaire or inventory method represents a step forwa~d in job analysis methodology. Alihough the military services have been usJng task inventories for some time (e.g., Christal, 1974), they are a more recent phenomenon for private employers and municipalities. A questionnaire format offers several advantages over the observe-and-interview technique. Once a I ist of tasks has been identified, objective data can be obtained and summarized for an extremely large proportion of job incumbents in a short time and with relative ease. This capacity for widespread sampling greatly increases the reliability of the job analysis findings. In addition; since task inventories are objective measures, they are extremely us~ful for making cross-jurisdictional comparisons. Three major trends suggest themselves in the use of job analysis checklists and inventories. ~irst, a number of researchers are using previously existing questionnaires such as the one developed by the California Bureau of Industrial Education and Division of Vocational Education. This eliminates a great deal of redundant effort, but unless the questionnaires used in different jurisdictions are identical, comparisons across studies are extremely difficult or impossible. A second trend is for researchers to develop new task inventories in each jurisdiction performing an analysis. This approach incorporates locally important tasks which may be omitted from an inventory developed in another locale, and· it ensures that departmental idiosyncracies in emphasizing different areas of performance wil I be reflected. Unfortunately, none of the researchers whose studies we reviewed suggested methods for scoring their inventories to allow inter-jurisdictional comparisons at a duty level. Instead, an item-by-item comparison, extremely tedious without access to automatic data processing equipment, must be made. The third trend we perceived in firefighter task inventories was the use of the Position Analysis Questionnaire. This instrument has been carefully developed on a variety of jobs, and objective job dimension or factor scores can be obtained inexpensively from a central scoring service. Furthermore, indications of desirable personal attributes, and estimates of General Aptitude Test Battery requirements for the position 19. are routinely provided. The PAQ's greatest weakness, however, is associated w·i·th its broad applicability. An instrument general enough to be relevant to all blue collar jobs may lack the specificity required to detect important differences across fire departments, and thus, to reflect differential job requirements. Furthermore, as the study by Cecil and Cecil demonstrated, the worker-oriented approach of the PAQ is not readily amenable to developing the task-oriented job performance dimensions most useful in an empirical validation. While it is generally considered desirable to employ more than one job analysis strategy in a study to obtain simultaneously the benefits of several methods, it becomes difficult to sift through the mass of job information which results, and to integrate the products of the various approaches. Researchers seldom identify their information needs in advance, and plan their job analysis techniques accordingly. A perhaps related problem is a fairly common failure to communicate critical information about the job analysis methods employ~d. This is frequently manifested in neglecting to describe exactly the procedures for the various tasks job experts have been requested to perform. Indicating that 11 supervisors and incumbents generated attributes required to perform each task'' provides 1ittle guidance to sonieone wishing to replicate a study. Similarly, failing to describe (or even develop) a rationale which underlies the procedures followed, at best impedes the development of the art of job analysis, and frequently produces confusion about exactly what the researcher is trying to do. Finally, in very few of the reports we read was any estimate provided of the extent of agreement found among raters, among juri sd i ct ions, or among different 1y sized departments. In a few cases, researcher reported that job analyses indicated a high degree of job similarity among several jurisdictions included in~ study, but the means of determining this similarity were seldom described. In general, information regarding the numbers and qualifications of job experts included in a study, the lengths and sizes of any workshops held, specific instructions given to participants, the rationale for any rating tasks, and the leveJs of interrater agreement should be provided whenever they are applicable. In part, ~e support the concept of unique, original, and individual approaches to job analysis. Yet for the sake of practicality and portability, some common firefighter job analysis instrument which could be easily administered and scored is sorely needed. Such an instrument, if reasonably brief, could form a common core in a large proportion of future ~tudies to permit generalizing local results to other jurisdictions, and yet would not be so consuming of time, interest, and money as to preclude additional explorations if local agencies desired. We str~ngly recommend developing such an instrument, and hope that the materials described in subsequent Sections of this report may represent progress in this direction. 20. Job Performance Measures The purpose of this section is to review the different types of job performance measures (c~iteria) used in firefighter test validation studies. Some of the 1iterature we reviewed was related to certification or training rather than selection. Other studies used a content validity design and no job performance mea~ures were constructed. Therefore, this section wi 11 deal primarily with criteria used in empirical validity studies. As is true of test validation projects for most occupations, the most frequently used firefighter job performance measure has been the rating scale. Brumback ct al. (1974) mention several other possible criteria. They indicate that various records including training grades, proba tionary evaluations, merit awards and discharges, service connected injuries, and turnover are possible sources of job performance informa tion. In a multi-jurisdiction project such as ours, however, records would be of iittle use due to lack of comparability across departments. Thus, carefully constructed simulations or rating scales seem to be the most promising approach. The Selection Consulting Center (1973) identified 12 performance evalua tion areas in their job analysis. Nine of these dimensions were cogni tive areas: Learning Abi 1ity, Mechanical Comprehension, Spatial Orienta tion, Construction Knowledge, Judgment, Following Procedures, MonitoringEquipment, Improving, and Mechanical Knowledge. The three noncognitivecriteria were Compatibility, Reasonableness, and Teamwork. Ratings on these dimensions were made using 9-point scales, anchored by five semibehavioral descriptions of different performance levels. At least two supervisors, trained by project coordinators, rated each firefighter.After making indipendent ratings, supervisors met to resolve rating differences and to complete a final consensus rating. sec did not obtain interrater reliability data nor do they report lntercorrelations among performance dimensions. The 12 criterion scales, although anchored by job-related statements, represent the kinds of individual differences constructs typically used as predictors invalidation studies. Thus, the SCC project is characterized by a trait-orierited rather than a task-or behavior-oriented approach to performance evaluation. Given the striking similarity between at least the titles of some criterion scales and some predictors, it would seem far more critical (and more difficult) to demonstrate the validity of the performance rating scales as measures of actual job performancebehaviors than to demonstrate high and significant correlations between predictor instruments and the ratings. 21. The staff of Personnel Decisions, Inc. recently completed three fire fighter validation studies. The first of these projects involved the position of firefighter in the Minneapolis Fire Department (Personnel Decisions, Inc., 1971). PDI developed scales in three stages. First, the captain in charge of training was interviewed to obtain a preliminary listing of important job dimensions. Researchers then held a workshop with several individuals who knew the job and asked them to suggest modifications in the dimensions and to write anchors to define various points along each of the scales. The anchors were reviewed and edited for grammar and spelling by staff members, and the scales were opera tional ized. The second project was conducted for the Grand Rapids, Michigan, Fire Department (Heckman, 1973a). In this study, Heckman developed perform ance measures during a series of four four-hour workshops, each attended by approximately 15 supervisors who knew the job of firefighter well. During the first workshop, Heckman trained participants to write criti cal incidents describing particularly effective or ineffective job behaviors on the part of firefighters. Near the end of the first work shop, Heckman conducted a general discussion of the firefighter's job and developed a tentative list of major job performance dimensions. During Workshop I I, supervisors continued to write critical incidents, and at the end of this session, the group finalized the list of job performance dimensions. A total of 396 incidents was collected during the first two workshops. PDI staff edited these incidents and presented them to workshop participants at the third session. Participants read each incident, rated the effectiveness of the behavior described in the incident on a 9-point scale, and allocated each incident to one of four teen firefighter dimensions. From the results of Workshop Ill, PDf developed a tentative set of behavior observation scales by identifying incidents which supervisors agreed represented a particular dimension and selecting items representing different levels of mean effectiveness with high inter-rater agreement. In the final workshop, supervisors reviewed the scales and rated several firefighters as a pilot test. Heckman (1973b) used a similar procedure in a validation project for St. Paul, Minnesota, firefighters to develop a set of performance dimen sions. The dimensions·developed by PDf in these three studies are presented in Table 2. Figure 6 shows the St. Paul firefighter perform ance scale "Taking Effective Action at Fires." Inter-rater reliabilities, estimated by intra-class correlations, were in the barely acceptable range for Grand Rapids, ranging from .21 to .54 across scal~s. Inter-rater reliabi l ities for the St. Paul scales ranged from .30 to .72. Heckman factor analyzed the performance dimension inter correlation matrix for the Grand Rapids data and identified three fairly distinct clusters of dimensi6ns. Cluster I was a technical, task oriented, firefighting factor, cluster 2 was defined by interpersonal and motivational 22. TABLE 2 JOB PERFORMANCE DIMENSIONS DEVELOPED BY PERSONNEL DECISIONS IN MINNEAPOLIS, GRAND RAPIDS, AND ST. FIREFIGHTER VALIDATION PROJECTS Minneapolis DimensJons Unverstandinq buildings, constructions, and fire behavior. Mechanical ability. Ability to profit from training. Flexibility when per forming different posi tions of a crew. Holding up under pressure and stress. Carrying out orders under firefighting conditions. Thoroughness in carrying 'procedures when sea~ch ing for hot spots. Getting along with other firefighters and employees. Showing teamwork. Amount of supervision necessary. Maintaining public relations. Overall job effective ness. Grand Rapids Dimensions Firefighting. Attitude. Building inspection. Desire to learn. Personal appearance and hygiene. Ability to learn. Courage. Mechanical work. Public relations. Driving and operating heavy equipment. Getting along with other firefighters. Report writ i nq. Strength and. endurance. Physical coordination. Overall job performance. PAUL St. Paul Dimensions First aid. Getting alonq with other firefighters. Driving and operating heavy equipment. Publ fc relations. Mechanical work. Taking effective action at fires. Attitude. Desire to learn. Personal appearance and hygiene. Ability to learn. Couraqe. Prefir~ planning. Being physically fit. Overall performance. TAKING EFFECTIVE ACTION AT FIRES 23. Using good judgment; rema1n1ng calm and quickly taking proper action at fires; knowledge of firefighting and use of appropriate firefighting behavior; observing and investigating the cause of fires; Ubing appropraite firefighting equipment in real or simulated emergency conditions; thinking ahead and taking the initiative. 9 8 A firefighter saw, identified, and reported electrical wires in contact with a steel fence adjacent to a fire scene so that pre cautions were taken to avoid injury to the nearby men. 7 At a house fire when all the other crew m~mbers were searching the interior for victims, the firefighter replacing the regular driver recruited help to raise ladders and effectively ventilated the house. 6 At an extremely smoky fire, a firefighter quickly a~d properly ventilated the roof. Arriving at a mattress fire, a firefighter jumped off the rig, grabbed an axe and pump can, and followed the captain to the fire scene. 5 4 The firefighter of the ladder company started upstatrs to a fourth floor fire, but forgot to bring the sprinkler shut-offs and had to return to the ladder truck. 3 Then a firefighter was ordered to place a smoke extractor in operation at a smoky house fire, he carelessly put the fan in a window close to an adjacent building so that the blowing smoke caused unnecessary damage. 2 At a second-story house fire, a firefighter forgot to secure the top of a ladder and climbed up with a charged 1-1/211 hose line. When he opened the nozzle, the backpressure caused the man, the hose 1ine, and the ladder to fal 1 away from the building. Figure 6. St. Paul firefighter performance rating scale (Heckman, 1973b). 24. dimensions, and cluster 3 reflected courage, strength, endurance, and physical coordination. Inter-rater rel iabi 1ities were higher for the clusters than for the separate dimensions, ranging from .58 to .79. A factor analysis of the St. Paul criterion data resulted in very similar clusters or factors, except that factor 3 for the St. Paul data consisted of First aid and Desire to learn. This imperfect agreement between the two studies probably arose because the dimension Being Physically Fit was omitted from the St. Paul study factor analysis. Murdy and Norton (1972) developed a set of job performance measures in connection with their firefighter selection study for the city of Fort Worth, Texas. These rating scales were developed from a comprehensiveexamination of the department 1 s efficiency rating instrument and through discussions with district fire chiefs. Murdy and Norton 1 s performance evaluation form consisted of 40 items forming three broad categories of fireman performance. The first category, involving Emergency Work, contained 18 objective descriptions of job behaviors related to combating and extinguishing fires, performing rescue work, and protecting life and property. The second category was cal led Support Functions, and contained 14 objective descriptions of job behavior generally dealing with station work, preparedness, and fire prevention. The last category was labeled Social and Personal Aspects and contained eight descriptions of behaviors reflecting factors of social interaction or personality important for functioning effectively as a fireman. Murdy and Norton obtained two ratings on each firefighter using a paired comparison rating procedure." The researchers eliminated six sets of ratings because of intra-rater incon sistency, and some ratings because of disagreement between raters. Theyconcluded that the final set of rating~ showed a high degree of reliabi 1ity.Performance scores for each ratee were obtained by combining the responsesof the two raters over the 40 rating items to come up with an overal 1 criterion value. It may have been more appropriate to study the dimensionality of the criteria before summing ratings across potentially corre lated items. Researchers from the Atl·anta Regional Commission (1974) used a combination of approaches to develop job performarice measures. They randomly selected firefighters from five participating jurisdictions to generate critical incidents of job behavior, collecting 600 behavioral incidents which described varying degrees of firefighter effectiveness. Initially, the researchers attempted to sort these incidents into dimensions derived from PAQ elements rated as important for the firefighter 1 s job, but were unable to do so. Their 15 final categories reflected the PAQ dimen sions to a ~oderate extent, but included additional dimensions which sorters perceived in the critical· incidents. The Atlanta Commission concluded that the dimensions matched those derived by Murdy and Norton (1972) in a separate study. Finally, the researchers grouped the 15 dimensions into four broader classifications shown in Table 3. 25. TABLE 3 BEHAVIORAL CLASSIFICATIONS OF ALTANTA COMMISSION JOB PERFORMANCE DIMENSIONS I. Job Knowledqe Specialized job knowledge Interest in learning Helping others learn I I. Interpersonal Relations Relationship with peers Relationship with the public I I I. Approach to Work Will inqness and eagerness to do his part of the work at the station Willingness and eagerness to do his part of the work at the fire Respect for property and equipment Professional ism IV. Job Performance Remains calm in personally demanding situations Ability to perform physically demanding tasks Ability to follow orders Responsibility for the safety of others Ability to make cortect decisions Ability to work effectively in a hazardous, unpleasant environment 26. The Atlanta Commission developed four performance evaluation techniques. The first measure began as a 152-item checklist of behaviorally-based statements obtained from the critical incident data. A random sample of firefighters from the participating jurisdictions were asked to evaluate their co-work'ers on a 6-point scale showing how applicable each statement was to a ratee. The 152 preliminary items were factor analyzed to create IS five-item oblique factor scales tapping the per formance dimensions. The second group of criterion instruments was written performance tests and included a general firefighter job knowledge test, a first aid test, and 12 11 tab tests11 • The job knowledge test was assembled from train ing and promotional tests used by participating jurisdictions and included items related to fire prevention, salvage and overhaul, suppression, rescue, and ventilation. The first aid test was developed by the Georgia Heart Association and dealt with cardio-pulminary resuscitation. The tab tests were paper and penci 1 simulations of a fire in a mixed commercial/ residential building. Examinees read a description of the problems,and then judged which was the best of a number of alternative strategies for each of several operations including rescue, ventilation, and fire suppression. This format was chosen for administrative ease, economical feasibility, and a high correlation with actual work simulations in the past. The tests are intended to measure a firefighter's ability to integrate situational information with job knowledge when solving a problem. The third evaluation method comprised ten work segments or simulation tests designed to measure performance on several of the performance dimensions. The simulations included demonstrating (1) knots and ropes, (2) one-man hose lay, (3) spotting of apparatus and hook-up of suction supplyhose, (4) one-man ladder hand! ing, (5) hand traverse across a 24-foot 1 ad de r , ( 6 ) ha n d I i n g Sco t t a i r p a k s , (7 ) 1 ad de r c 1 i m b w i t h we i g h t , (8)ladder descent with weight, (9) improper and dangerous equipment storage,and (10) hose and nozzle inspection. Examinees were scored for appropriate technique, number of errors, and time required to complete each demonstra tion. In the final evaluation l';lethod, fi.re department records were perused for information about absences, injuries, commendations, complaints, disciplinary actions, and other performance-related ~aterials. The researchers report that white firefighters scored significantly higher on seven of the 23 objective performance tests, although theyprovided no. data nor sample sizes for the racial comparisons. It was concluded, however, that most racial criterion differences were attributable to differential job experience, not to actual performance 27. differences. When peer and supervisory ratings were averaged over the 15 per.formance dimensions, no statistically significant differences between blacks and whites were found. Both peer and supervisory ratings suffered from ceiling effects and halo error, althou~h supervisors wer~ lesser offenders. This is unfortunate, since peers• greater opportunity should increase the accuracy of peer ratings. Possibly, firefighters doubted the confidentiality and anonymity of the ratings and gave uniformly high ratings to avoid interpersonal conflicts or potential sanctions against those who might be rated lower. Supervisor and peer ratings correlated only .38, showing relatively poor agreement across rater types. \Jritten job knowledge tests and simulation tests showed greater variability and lower intercorrelations. Unfortunately, the rating data correlated relatively low with the other more objective criteria; and interestingly, the work history information taken from department files correlated essentially zero with both rating and test measures. The Atlanta Regional Commission performed two factor analyses of their performance data. Only two factors were extracted from the peer ratings, indicating a strong halo effect. A separate analysis of the composite supervisory rating, scores on the p~rformance tests, and the archival work history data produced strong factors defined as strength and agility, equipment problem solving, job safety, and job knowledge. In addition, a number of unusual bipolar factors appearerl, defined by performance measures which apparently represented opposite extremes of some not readily identifiable continuum. It is dangerous to include uninterpretable factors in validation research especially when a high score on one ''favorable11 attribute necessari Jy implies a low score on another as is the case with bipoTar Factors. The Atlanta researchers concluded that the performance factors emerging from the two analyses tapped alI the important dimensions identified in their job analysis, except motivational and interpersonal areas (which actually were included in the supervisory and peer ratings). They emphasized that their multidimensional results preclude using a single type of selection instrument to predict firefighter performance adequately. To summarize, although parts of their report were not completely clear, the work performed by the Atlanta Regional Commission represents a tremendous effort toward developing sound measures of job performance. The use of job knowledge tests, tab tests, and job simulations as criteria certainly deserves further investigation. Since the psychometric properties of the job knowledge and tab tests were not fully reported, further research on 28. the data collected by the Atlanta Regional Commisson would be extremely advisabl~. The simulations should also be evaluated for inter-rater rei iabi lity, and to assess the effectiveness of the standardized adminis tration instructions. Certainly the researchers involved with this project are to be commended for seeking unique, albeit time-consuming procedures for measuring job performance of firefighters. Further analysis of the strength of relationship between peer and supervisory evaluations by dimension, also seems warranted. ' Ceci I and Ceci I (Note I) followed an approach simi Jar to that used by the Atlanta Regional Commission. Based on a job'analysis, the researchers constructed 23 performance measures to evaluate firefighter effective ness. Twelve measures tapped job ski 1ls in 12 performance stations at the fire department training school. Three officers and one firefighter formed a rating team. The tasks included salvage operations, generatorsand 1ights, operation of fire apparatus, maneuvering ladders, basic hydraulics, rescue operations, hose practice, rope practice, aer:al ladder practice, first aid practice, breathing apparatus, and radiological monitoring. The researchers do not report inter-judge rei iabi 1ities but indicate that the senior experimenter observed the rating processand conducted spot checks to assess the degree of agreement. The second type of performanc_e measure consisted of paper and penci 1 jobknowledge tests, taken from Montgomery, Alabama, fire department training materials and operating procedures, and from the IFSTA manuals. Are?s covered by the knowledge scales included organization and chain of comman~, rules and regulations, communications and signals, forcible entry, ventilation, basic fire chemistry, and sprinklers and standpipes. The researchers also developed rating scales for four attitudes:· job interest, adaptation to changing conditions, dependability, and getting along with others. Each firefighter was rated independently by three officers on these attitude scales: The researchers computed the total rank order of the 69 participating firefighters by summing scores across all dimensions and all performance measures. They also developed scbres for the three main divisions of the rating instruments, i.e. performance, knowledge, and attitude. Tice (1970) examined the validity of the Iowa State University firefighter recruit battery performing a concurrent validation using 300 firefighters from four departments. Tice reported high coefficient alphareliabilities for his peer nomination criterion instruments. He also developed a job behavior check! ist to evaluate firefighters, employing Smith and Kendall 1 s (1963) retranslation procedure. Test-retest reliability for the check! ist was .69. In addition, Tice used absenteeism ~nd a job behavior ch~ckl ist for supervisors to rate firefighters. He does not report the inter-relationships among criteria. 29. Finally, Ruedebusch et al. (1975) used a modification of the critical incident approach to develop behaviorally anchored rating scales. T~ey collected 65 critical incidents during their job analysis phase. The researchers independently sorted the incidents into subject matter categories, then met to discuss the resulting dimensions and arrived at seven major performance areas: (1) ability to learn and apply learning, (2) judgment under stress, (3) physical fitness, (4) compatabil ity, (5) public relations, (6) teamwork, and (7) pride and dedication to career. Ruedebusch et al. presented the dimeniions to Des Moines firefighters and officers and asked them to write additional performance examples, collecting an additional 425 usable incidents. Twenty-three firefighters and 50 officers in three cities independently assigned each incident to one dimension, and incidents with at least a 65 percent agreement in categorization were retained. The researchers edited the incidents, eliminated duplicates, and grouped the incidents by dimension (at least 20 per dimension). Firefighters and officers from three cities then assigned a scale value from one to nine to each incident to indicate the level of effectiveness or ineffectiveness of each performance specimen. The researchers retained incidents for which at least 60 percent of the raters gave the modal value, and incidents with standard deviations of less than 1.70. They then wrote additional items to fill gaps at the midpoints of the behaviorally anchored scales, and repeated the sealing procedure, retaining items as indicated above. Finally, they wrote three additional scales, two measuring fire fighters• overall performance in emergency and in routine daily activities, and a third scale providing an overal 1 performance estimate. 'Fireftghters who were tested with an experimental battery also rated themselves on the same rating forms used by officers. Inter-rater rei iabil ity coefficients between self and officers were quite low for Omaha (ranging from .08 to .19) and for Des Moines (.08 to .28). Only officer ratings were retained for the empirical validity portion of the project. To explore the dimensionality of the supervisory ratings, the researchers factor analyzed the rating scale correlation matrix using Kaiser 1 s Little Jiffy factor program and an oblique rotation. Two very highly correlated factors emerged. The first factor seemed to measure the application of learning and judgment aspects of the job, and the second was an inter personal and motivational factor. The researchers constru~ted a third overall measure by summing all rating scales since the intercorrelation of the two factors was quite high (r.=.90; neither the intercorrelation matrix of individual performance dimensions nor the factor loading matrix suggested that the intercorrelation was as high between the two factors as reported.by the researchers). . In summary, our review indicates that some promising developments in per formance measurement have occurred. Construction of job knowledge tests, job samples, and simulations represent encouraging trends. Although 1n many situations the cost of these approaches can be prohibitive, Wernimont 30. and Campbell (1968) and the work by J. T. Campbell ( 197.2) strongly encourage using alternative ways of measuring job performance. Combin ing the various elements of job performance research described above would probably lead to an ideal model for selection research and criterion development. Unfortunately, most of the studies we reviewed contain either serious methological flaws, incomplete or inaccurate reporting, or only cursory examinations of the interrelationships and psychometric properties of their performance measures. Often researchers made arbi trary judgments about how to combine performance dimensions. Important characteristics of the performance measures were often omitted from tables and statistical presentatio~ of results. Departmental records do not present much promise, judging from the research cited above. Multi-jurisdictional approaches obviously preclude the useof such departmental records. More work is needed on developing sound simulations and job samples. Too often researchers assume that a job sample is a valid and reliable measure simply because it seems related to the kinds of tasks performed by firefighters. Seldom were job samples and simulations rigorously evaluatedto test their own content validity as job performance measures. Peer ratings and self ratinqs have been widely used by firefiqhter selection researchers. Recent research conducted by Borman (1974) sugg~sts that peers may be in the best position to evaluate some aspects of job performance, while supervisors may more accurately and reliably estimate other aspects. Clearly, more research is required on this topic in firefighting and in other occupational areas. The above research has been extremely helpful for .purposes of the present project. We have learned that for the position of firefighter variousrater errors such as halo, leniency, and central tendency errors aredifficult to overcome. We are familiar with the numerous methods formeasuring job performance, and with the difficulty of constructingreliable, accurate, est.imates of firefighters• performance. Selection Proced~res The lack of common job performance dimen~ions across studies makes itdifficult to generalize validity findings, even when the same predictorshave been used in different departments. However, where there are somesimilarities in the two areas, we shal 1 attempt to choose predictors which have ~orked wei I for other researchers. 31. Selection Interview. We found no empirical research testing the validity of the selection interview for firefighter applicants. The Selection Consulting Center (1975) did, however, make several recommendations for conducting such interviews, which could be useful to Civil Service or fire department screening boards. sec identified four performance areas amenable to probing by interview, and asked Civil Service or fire department representatives from 53 cities to write interview questions tapping these areas. The questions which were submitted were edited and combined to develop an interviewing guideline and scoring technique, assessing 17 specific attributes in the four broader dimensions (see Figure 7). The SCC intervi~w recommendations focus on identifying the applicant's behaviors in earlier situations similar to those he would face as a firefighter. For example, to predict behavior in stressful situations, SCC recommends asking questions such as, "Describe any dangerous, personally hazardous, or emergency situation you have been involved in. How did you respond?" One California department tested a modified version of the SCC inter view process in screening a large number of firefighter applicants. The results suggested that inter-rater reliabilities can be Impressive when rating scales are carefully defined and when interviewers are thoroughly trained. Clearly, however, more research is needed on the validity and reliability of interviews for screening firefighters. Physical Ability Testing. Physical ability testing practices today appear to 1ie on a continuum representing similarity to the actual job requirements faced by firefighters. At one extreme, experts judge what kinds of movements and exertions firefighters are required to perform, and create or select variou·s calisthenics or other abstracted exercises which require the same or similar abilities. At the other extreme, applicants are dressed in turn-out gear, and are made to run, jump, and climb while 1ifting and carrying equipment or weights. The degree of similarity to actual tasks, and the amount and quality of evidence tying abstract exercises to firefighting requirements are the two major parameters r~qui red to assess the content validity of such physical seleciion methods. Verducci and Meekins (1973) surveyed 22 cities and a number of profes ·sional organizations to determine what kinds of physical aptitude tests existed for fire service applicants. Of six cities and four professional firefighter groups that responded, only two cities and two pro~ fessional organizations had developed scorable physical tests, usually comprising ~everal calisthenic exercises such as pull-ups, sit-ups, and standing broad jump, and some form of obstacle course or agility run which in some cases was made to simulate actual firefighting duties. Verducci and Meekins next surveyed San francisco firefighters to identify 32. ATTRIBUTE I. Communication Skills Persuading Oral Communication Teamwork II. Interpersonal Activities Compatabil ityInterpersonal RelationsFollowing Orders AdaptabilityInnovation Ill. Job Orientation Interest Pride Motivation to Learn Motivation to Advance Personal SacrificesCoping with Human Suffering IV. Maturity Tolerance for Stress Tolerance for Boredom Clean! iness Figure 7. Job performance dimensions tapped by interviews, Selection·Consulting Center. (1975) 33. to what extent firefighters perform various physical skills, such as pushing or pulling objects, 1ifting or lowering objects, and carrying objects. Each of these skills was rated by a kinesiologist, by two fire officials, and by the two authors, physical educators, to determine for each the importance of nine components of physical movement (e.g. coordination, stamina, explosive strength), and of nine body parts (e.g. hand and wrist, low back, upper leg). Physical abi 1 ity tests were then assembled to tap these movement components in the body parts, and were weighted in proportion to their relative occurrence in firefighting, estimated from the San Francisco survey. The tests included: (1) a bi-manual coordination test of attaching an~ removing threaded nozzles; (2) bent-knee sit-ups; (3) hand-grip strength test; (4) a strength and balance test, carrying a 125-pound weight up and down stairs and along a weaving course for time; (S) a bending and twisting exercise; (6) pull-ups; and (7) a 500-yard shuttle run. The physical test was validated for a group of 14 firefighter recruits. Four training instructors made pair comparison ratings of these recruits' physical abilities, and the ratings were correlated with scores on the. physical ability test. The test's validity was .68, and is statistically significant eVen with the s~all sample size involved. In a second analysis, 635 applicants including 52 blacks, 38 Spanish Amer1can, 1 I oriental Americans, and 11 other minorities were tested on the physical ability battery. No racial differences in mean test scores were obtained for any of the individual exercises or for the total combined score. These results are extremely encouraging, and clearly show the value of physical abi 1 ity examinations. The Verducci and Meekins study is a landmark project in carefully developing and validating firefighter selection standards. The approach used by SCC (1975) to develop a suggested physical performance exam was in some respec~s similar to that used by Verducci and Meekins. sec performed an extensive 1iterature review to identify firefighter tasks and reviewed the physical performance literature to determine which physical skills were involved in performing each task. Five broad dimensions of physical abi 1 ities consistently emerged as relevant to frrefighter duties: coordination, balance, endurance, strength, and flexibility. Four hundred three firefighters from a number of departments rated the relative importance of each of these five skill dimensions for performing each of 64 tasks. Of the five factors, coordination and strength were rated as most important for performing the physically-oriented tasks included· in the questionnaire. The other three dimensions received approximately equal weight and were rated as slightly more than half as important as strength and coordination. • ' 34. sec helped several fire departments develop physical performance examinations during the course of their project. A sample examination, used by the University of California, Davis, fire department contains the following types of items: equipmen.t removal, stair climb, hose hoist, wet hose load, advance hose line, climb, crawl, drag, and joist walk. Romashko, Brumback, Fleishman, and Hahn (1974) describe a possible physical testing procedure. Based on their job ~nalysis of New York City firefighters, they recommended an experimental battery tapping eight ability areas. The abilities required and tests selected to measure these abilities are presented in Table 4. The majority of these tests were selected on the basis of factor ana lytic~! research conducted by Fleishman (1964). The researchers pro vide an appendix to their report with complete instructions to candi dates and examiners for all tests and present means and standard deviations for a sample of 23 firemen who took these tests. Ruedebusch and Keck (1975) had a physical education expert review their firefighter job analysis results, estimate the physical demands placed on firefighters, and ~ssemble a series of physical ability tests to reflect these demands, and to simulate as nearly as possible actual firefighter tasks. The resulting battery included: (1) ladder climb; (2) dummy drag; (3) walking balance beam; (4) stooping, kneeling, crawling; (5) squat thrusts; (6) crawl through a dark room. Despite the low v~lues reported by Verducci and Meekins, our own experience suggests that a majority of fire departments in the U.S. require at least some demonstration of physical ability from applicants, although the skills required may vary widely from department to depart ment. Because the Verducci and Meekins study in San Francisco was so carefully performed, and because it was funded in response to 1itigation, other departments are likely to adopt its recommendations and assume its validity wi 11 generalize across jurisdictions. Careful and thorough research like Verducci and Meekins 1 should be performed wherever possible, however, since physical abi Jity testing has the greatest potential for assessing directly qualities demonstrably required for firefighter performance. Professional research in several locations around the country could go far in discovering whether valid, uniform national norms for physical standards can be established. The most promising approach toward developing and validating physical performance measures is to creat~ job simulations that closely para! lel tasks performed by firefighters. Where more abstracted physical performance measures such as push-ups and chin-ups are used, expert raters should be used to docu ment the connection between job tasks and physical performance requirements . • 35. TABLE 4 ABILITY TESTS RECOMMENDED BY AIR FOR NEW YORK CITY FIREFIGHTER SCRFENING Ab i 1 it i es Tests l. Stamina l. One-m i 1e Run 2. Static Strength 2. Five-minute Free Style Stepping 3. Hand Grip Preferred 4. Hand Grip Nonpreferred 3. Explosive Strength 5. Free Style Broad Jump 4. Gross Body Equilibrium 6. Balance 5. Extent Flexibility 7. Twist and Touch 6. Gross Body Coordination 8. Cable Jump 7. Dynamic Strength 9. Leg Lifts Trunk Strength(Dynamic Strength) 8. Dynamic Flexibility 10. Bend, Twist, and Touch 36. Paper and Penci 1 Tests. The two approaches most often found in thefirefighter selection literature for developing written exams are contentvalidity which follows a rational test construction model and empiricalvalidity, in which test scores are correlated with job performance measures. An example of the former approach is the exam suggested forthe Fort Worth Fire Department by the Psychological Services Branch,Dallas Region, Civi 1 Service Commiss1on. Although the researchersdid not actually construct an exam, they provide very helpful suggestionsand an examination plan,which could be operationalized. FollowingPrimoff 1 s Job Element Approach, the researchers identified 13 importantjob elements or crucial areas for testing. These elements were presentedin Figure 5. For each major element they 1isted sub-elements and suggestedappropriate measures. The researchers suggested assessment centers,written or oral tests, and strength and agility tests. They correctlyindicated that the more abstract selection variables, such as personalattributes, require more extensive empi rica! evidence. Space does notpermit us to describe in detail the examination plan, but a few examplesmay give the reader an appreciation of the approach which was followed.The job element 11 Abi 1ity to work with minimum supervision11 had a numberof sub-elements, which the researchers suggested could best be measuredthrough an assessment center; On the other hand, mathematical abi 1itycan be best tapped in paper and penci 1 tests. Certain sub-elements under mechanical abi 1ity can be measured by written test items, but the majorityof them would best be tapped by work sample or simulation exercises. After considering several types of job analysis information, Brumbacket al. (1974) formed a test plan showing significant job elements, andmeasurable constructs likely to tap,them. The written test constructs included understanding job information, reasoning and problem solving,dealing with others, understanding mechanical devices, and spatialorientation and visual memory. Since aptitude, personality, and interest measures are not as readilyvalidated with content validity approach as are knowledge and skills,a large number of empirical validity studies have been performed forfirefighters in the last several years. Virtually al 1 of these havebeen concurrent rather than predictive studies. Heckman (1973b) performed a concurrent study in St. Paul, Minnesota,using the California Psychological Inventory (CPI), Gough AdjectiveChecklist (ACL) and Minnesota Vocational Interest Inventory (MVII)as predictors. He obtained a cross-validity estimate of .37 for acomposite o~ the CPI Responsibility and Intellectual Efficiency scales,and the MVI I Warehouseman scale predicting a composite of job performancedimensions dealing with operating apparatus, mechanical work, takingeffective actions at fires, ability to learn, displaying courage, and 37. prefire planning. A second composite of the CPI Self Control scale, and the ACL Self-con~idence and Affiliation scales had a cross-validated correlation of .31 with a cluster of performance dimensions relating to interpersonal aspects of the firefighter's position. Finally, the CPI Communality scale, the ACL lntraception and Abasement scales, and the MVI I Hospital attendant scale had a .37 cross-validity for a combination of the First aid and Desire to learn performance dimensions. Cognitive and mechanical comprehension paper and pencil tests and b(ographical history questions had no validity for any of the criterion composites in this study. In Grand Rapids, Michigan, Heckman (1973a) found that the Restraint scale of the Guilford-Zimmerman Temperament Survey and the ACL Succor ance scale had a mean cross-validity of .32 with a composite of per formance dimensions related to dealing effectively with others. In addition, the MVI I Plasterer and Carpenter scales correlated .21 (mean cross-validity) with a "firefighting and operating apparatus" performance cluster. Here, too, tests of verbal comprehension, mathe matical reasoning, space visualization, and mechanical comprehension yielded no significant correlations with any performance measures. Among the more interesting finding~ resulting from the Minneapolis, St. Paul, and Grand Rapids firefighter validation projects conducted by PDI is the consistency of results across studies for certain pre dictors. For example, the Autonomy scale of the ACL correlated nega tively with a dimension which cut across all three studies involving getting aldng wi~h other firefighters, while the ACL Deference scale correlated positively with the same criterion dimension. For the Mechanical Ability performance dimension, the MVI I Stock Clerk scale correlated negatively in al 1 three studies with the validities ranging from -:14 to -.22, while the MVI I Sheet Metal Worker scale correlated positively (.23 and .28) in two of the studies, although the correlation was essentially zero for the St. Paul sample. Murdy and Norton (1972) used a number of Employee Aptitude Survey (EAS) tests, the Bennett Mechanical Comprehension Test, and Eysenck Personality lnven-· · tory, the Cattell 16 Personality Factor Questionnaire (16PF) and a series of personal history items in their validation project for Fort Worth fire fighters. The criterion measure in this study .consisted of pair compari son ratings on each of 40 firefighter duties or skills, summed across two raters and over the 40 skills. The EAS Visual Pursuit and Symbolic Reasoning tests correlated .28 and .23 respectively with the summary criterion score. Four personality scales correlated significantly with the performance measure: Extro version vs. Introversion (r=.l7); 16PF 0 (Apprehension and depression; 3R.. r=.20); l6PF0_2 (Self-sufficient, resourceful; r=.l7); and 16PF03 (Self control; r=. 18). Finally, a composite of ten biographical history items, each significantly related to criterion scores, had a validity of .21. Murdy and Norton performed stepwise multiple regression analyses to com bine all 9 aptitude and all 20 personality scales, and performed a secondlevel multiple regression to combine these two composites with the biographical history scores in predicting total performance. Stepwise regressiontechniques are escpecially likely to capitalize on sampl~-specific variance in weighting variables, and hence the lack of a cross-validation estimate,· and the extremely lovv foldback values of .32 and .43 for the aptitude and personality composites respectivefy raise some question of the value of their predictor battery. The second-level foldback coefficient obtained by regressing the three predictor comnosites onto the oriainal performance scores (thus maximizing the relationships between performance scores and . three composites preselected for their high criterion correlations) is only .54, again a r~latively low value considering the opportunity for optimizing relationships present in the sample only by chance. The .seven ·zero-order correlations described earlier (two aptitude, four personality, and one biographical) give a much more reliable estimate of the strength of the relationship between the test battery and firefighter performance; these values cluster very close to .20, although summing these seven scales could probably be expected to produce a composite correlation on the order of .40 to .50. Ruedebusch et al. (1975) report that only one variable from the Person ality Research Form (PRF) correlated significantly in both the Des Moines and Omaha firefighter samples. The Autonomy scale correlated with the job performance dimension of Overall emergency. Ruedebusch et al. also report nonsignificant relationships between the IPMA Firefighter Test and criterion measures of learning ability. They did, however, have some success with item analyzing the PFR in Des Moines, and applyinq the scoring keys to an Omaha cross-validation group. Cecil and Cecil (Note 1) used the Bennett t~echanical Comprehension Test, the Minnesota Paper Form Board, the Flanagan Aptitude Classification Tests, and the Vocations Preference Inventory described by Holland (1965) to predict a composite of rating and simulation exercise scores in a sample of 72 subjects which cut across all ranks of the Montgomery, Alabama, fire department. Cecil and Cecil obtained a foldback multiple correlation of .58 using the Bennett and the inspection test of the FACT series. Although they did not validate their recommended selection battery, the Atlanta Regional Commission (1974) reported that the tests being used by the jurisdictions in their study were minimally significantly related to some dimensions of job performance. t!o description of these tests was provided, however, and the researchers failed to note that prior selection on the existing test batteries would severely restrir.t the range of scores, and hence greatly attenuate the validities of these tests..The Atlanta researchers recommended a new testing system designed to tap the characteristics required to perform well in their job performance dimensions. These characteristics include mental .abi 1ity, perceptual speed and accuracy, field perception, reasoning ability, spatial relations, mechanical abi 1ity, interest, biographical informa tion, and physical fitness. Tice (1970), in a study assessing the validity of. the Iowa State University firefighter recruit selection battery, obtained cross-validated correlations of .30 between a composite of three Guilford-Zimmerman Temperament Survey (GZ) scales (Objectivity and Restraint weiqhted positively, and Emotional Stability weighted negatively) and an overall performance measure based on peer nomination ratings on seven performance factors (building morale, speed and precision in emergencies, technical knowledge, getting alonq, most desirable co-worker, best source of technical advice, and best all around firefighter). Unfortunately, no significant correlations were found between any predictor scales and three other criteria (absences, peer and supervisory ratings on a set of behaviorally anchored rating scales). The 11 nonsignificant11 cross-validities Tice did obtain (.20 between peer ratings and two GZ scales, and . 12 between absenteeism and a composite of three GZ scales and the Gamma test of the Otis Mental Ability Test) seem relatively large considering that the sample had been preselected on both the Otis and the Guilford Zimmerman scales. The resulting restriction in range could be expected strongly to attenuate the cross-validities Tice reported, making the predictors appear to perform much worse than was actually the case. The Selection Consulting Center (1973) reported on a concurrent val ida tion study involving jurisdictions in California and Nevada. They used the Firefighter B-1 (P) Form prepared by the International Personnel Management Assoc1at1on (IDMA), the Fire Aptitude Test Form 45 prepared by cooperative personnel services, Cal itornia State Personnel Board (CPB), the Oral Directions Test Form S/T published by the Psychological Corpora tion. All three cogn1tive predictors demonstrated validity for whites (N=390), blacks (N=41), and Spanish surnamed (N=34) samples for an overall criterion composite. The validities of these three tests for whites ranged from . 16 to .25. The validities for blacks ranged from a high of .58 for the Oral Directions Test to a low of .36 for the IPMA test. The val idit1es for the Spanish surnamed sample were .47, .52, and .57 for the oral directions, IPMA, and CPB fire aptitude tests respectively. Although whites scored higher on all three tests than 40. the two ~inority groups, the sec qroup found no different·ial validity usinq an analysis of covariance, and concluded that a sinqle cutting score is appropriate for all applicants on each exam. In an unpublished study, Rusmore (~ate 3) reviewed the six performance dimensions and the 20 most important tasks identified in the Selection Consulting Center study and concluded that mechanic'al ability made a substantial contribution to performance in these areas. To test this hypothesis, he corr~lated scores on the Fire Aptitude Test (FAT) Form 40 a measure of fire service-oriented mechanical and mathematical abilities,with probationary and one-year performance ratings for two qroups of firefighters. The correlation with probationary ratings for a aroup of 76 trainees was .36, and for.a subset of 29 of these firefiahters, the correlations with 12-month job ratings were .43 and .40 for two rating scales developed by SCC which assessed mechanical skills and knowledge. Since all these experimental groups had been preselected on the basis of their test scores, the significant validities reported should be considerably larger for an unscreened applicant aroup. After correcting for the restriction in the ranqe of the developmental sample's test scores (relative to the scores for the total applicant group from which they were selected), Rusmore estimated that the validity of the FAT for predictin9 probationary ratings in an unscreened sample was .65, and its estimated validities for the two mechanically oriented 12-month performance rating scales averaged .72. Marks (1970) assessed the validity of the Personal History Index (PHI), a multi-factor background information questionnaire developed by the University of Chicago Industrial Relations Center, for predicting an overall performa~ce ranking. Firefiahters in ten California departments were rank ordered for "vocational success" by their respective traininq officers, and the top and bottom 27 percent for each department were identified, enc~mpassing a total of 215 persons. Ouestionnaires were mailed to these firefighters, and a total of lOR (60 hiah-and 48 lowsuccess) returned usable answer sheets. After controlling for aae and tenure in the groups, Marks found that three PHI factors sianificantly discriminated between the top and bottom ranked 9roups: Early Family Res pons i b i 1 i ty, Parenta I Fam i I y Adjustment, and the sum across a I 1 eight factors. Unfortunately, this high-low comparison method gives no indication of the scales' utility in the middle ranaes of the performance distributions, makina their value and fairness questionable in an actual selection situation. The majority of the empirical validation studies we have discussed share a weakness:· They' typically employ multivariate re0ression procedureswith inadequate sample sizes, and fail to cross-validate the results. 41. l·he results of al 1 such studies must be rejected unti 1 it is demonstrated that any favorable results are not merely due to factors specific to the developmental sample, and will actually generalize to an independent group. When zero order validities are examined, most significant correla tions are in the .25 range. Results of mental abi 1ity tests and mechanical ability predictors have been mixed, probably in part because of differences in criteria, restric tion in range for both predictors and criteria, differences in sample selection procedures, and the varying research designs employed. The majority of studies which we reviewed indicate that certain cognitiVe and mechanical predictors deserve further research, and virtually all rationally developed examination plans suggest the use of both cognitive and mechanical abi 1ity tests. Investigating the validity of cognitive and mechanical ability selection instruments can best be accomplished in a predictive validity design where there is no restriction of range due to preselection on the tests themselves. Typical behavior measures have also produced equivocal results. Although Heckman has had some success across departments with the Minnesota Vocational Interest Inventory and Gough Adjective Checklist, his results should be replicated by other researchers. A variety of other instruments including the Guilford-Zimmerman Temperament Survey, Cattell's 16 Personality Factor Questionnaire, the California Psychological Inventory, and the Personality Research Form have been employed as experimental predictors for firefighters. Perhaps an examination of the constructs underlying these measures, and an analysis of the intercorrelations -of the various personality instruments could be related to the empirical results to determine which types of scales have been successfully validated. For example, the Autonomy scale of the Personality Research Form and the Autonomy scale of the Gough Adjective Checklist have produced similar results in several studies (PDI, 1971; Heckman, 1973a, 1973b; Ruedebusch et al., 1975). Both theoretical support and multijurisdictional empirical validity must be established for personality or telllperament characteristics as prerequisites for predictors of firefighter performance. Until this kind of research is performed, our knowledae of the personal attributes required by fire service work wi 11 continue to be quite limited and conjectural. Similar 1imitations apply to biographical or personal background items. A subset of _promising b~ographical history items should be administered to firefighters in a number of jurisdictions. The items should be factor analyzed to determine underlying dimensions, and factor scores should be related to comparable criterion dimensions across several studies. The Personal History Index of the University of Chicago has already been factor analyzed and merits further investigation. 42. The West Valley Community Colleqe (1974) has developed a prom1s1ng approach to firefighter selection. This group employed a laboratory assessment center for firefighter screening for the city of Palo Alto, California. They developed assessment procedures based on the six dimen sions identified by the SCC job analysis. The individual exercises include a five-minute oral presentation; assembling a fire hydrant, hoses and nozzles; a multiple choice test on the identification of tools; and a written report. Group exercises in~lude small qroup interaction, a nonstructured group discussion requiring solutions to a number of prob lems, and a leaderless group assigned to reproduce a model. Although the investigators do not explicitly relate the assessment center exer cises to specific aspects of firefighter performance, their pioneering effort in this area is commendable. In summary, we were able to identify prom1s1ng leads for the test develop ment and validation phases of our project from the 1iterature review. Because of the problems of small sample sizes and lack of replication,multijurisdictional approaches are certainly called for in future investi gations of the validity of firefighter screen ina procedures. Although more carefully designed studies may not lead to significantly highervalidity coefficients, at least the level of confidence which can be placed in the findings of such studies will be much hiqher. The overall quality of selection research for firefighters is probably best reflected by the fact that very few if any studies have been published in the professional 1 iterature. We definitely see a need both for higher st~ndards and for more vigorous research in this area. Roles of Personnel Agency and Appointing Aqency. In a publication entitled Achieving job related selection for entry-level police officers and firefighters, the U. S. Civil Service Commission (1973) emphasizes the need for coordination between fire departments and government personnel agencies.Obviously, close cooperation is required to ensure that the list of eligiblecandidates reflects the needs of the department. To this end, fire service personnel should have input into the development and application of selection devices. At the same time, hovJever; a degree of independence is requiredbetween the merit system screening process and the appointment process.Thus, the Civil Service suggests that fire officials ultimately responsible for choosing recruits from an eligible list should not be directly involved in screening applicants for basic eligibility. The ideal combination is for fire and personnel officials to work together in developing objectiveselection procedures linked with subsequent performance, for the personnel agency to maintain primary responsibility for administering these procedures,and for the. fire service agency to select its recruits from amonq the candidates successfully passing this screening. 43. I I I. ORGANIZATION 0~ THE JOB A~ALYSIS REPORT The remainder of this report describes the methodology and results of our in-depth selection-oriented job anlaysis for the entry-level firefighter position. The report closely parallels the project phases comm~nicated to participating fire deoartments. We describe the samplingplan in Section IV of the report. Section V outlines the methodologyfor identifying firefighter tasks and for constructinq a preliminary task analysis checklist. We present the detailed findings of the mailedadministration of the task analysis checklist in Section VI. In Section VI I, we describe the procedures we used to develop homogeneous clusters of tasks. The structure and results of workshops designed to generatejob related information about the task clusters are described in SectionVI 11. Section IX summarizes results of a mail-out questionnaire designed to elicit additional information about the task clusters such as task importance and difficulty: knowledqes, skills, abilities, and other charac teristics required to perform these clusters; and behavioral examples of effective and ineffective perf~rmance. Sections X and XI discuss the implications of our findings for developing selection proecedures and for using our job analysis results in other _jurisdictions. Our summary and conclusions are outlined in Section XI I. IV . SAMPLE Our major sampling concern was that the departments in our study be nationally representative 'to assure the widest possible applicabilityof the results to individual jurisdictions. USCSC suggested that the sample include a minimum of 12-18 local jurisdictions, representing large, medium, and small fire departments from each of six areas or regions of the United States. Since we intended to collect most of our job analysis information by mailed surveys, we decided to increase the sample size suggested by USCSC. We also wished to obtain a certain amount of 11 independence11 between certain phases of the project. For example, we wanted to develop a 1ist of firefighter tasks using one group of cities which would be nationally representative, and then use an independent group of cities to rate the importance of and time spent on each task and to add additional tasks not covered by the checklist. We also wanted to request firefighters and supervisors in one group of cities to generate behavioral examples of job performance and to have participants from an independent group of cities categorize and rate the effectiveness of these example~. We feel that our ultimate plan provided us with continuity but also guarded against bias which could occur if only a smal 1 number of departments participated in all phases. Finally, for certain phases of the study, we felt it would be desirable to have a number of departments from each region and size eel 1 so that we could assess the level of agreement for departments within the same eel 1 as wel 1 as test for differences across cells. Our sampling plan for major project phases is provided in Table 5. A total of 126 jurisdictions was scheduled for the project. We selected cities from six major geographical regions of the country: Northeast, Southeast, North Central, Southwest,· West Coast, and North1vest, (see Figure 8). A fire department was classified as small if it was located in a city with a population of Jess than 50,000 and if there were fewer than 100 uniformed personnel in the department. We classified a depart ment as medium size if its city was between 50,000 and 100,000 in popula tion and the department consisted of more than 100 but less than 300 personnel. We classified a department large if the city population was over 100,000 and ·the fire department had over 300 personnel. We soon realized that developing a sampling plan was much easier than selecting specific cities to participate. ~e contacted the International Association of Fire Chiefs (IAFC) to obtain a complete listing of all fire departments in the country. After discussions with IAFC and other professional organizations, however, we learned that no complete listing of all fire departments existed. We were able to obtain a list of 86 large metropolitan cities from IAFC, including cities with a population of 200,000 and a firefighting force of 400 or more. The :ist was helpful but it did not provide us with the total population of cities from which we would select our sample. 45. TABLE 5 MAJOR PROJECT PHASES AND PARTICIPATING JURISDICTIONS Phase I II I I I Region Site Visits Generate Ta to sks Quto estionnaire P.ate Tasks Regional Task C 1 us t e ring Workshops s M L s ~~ L s M L 1 2 3 4 5 6 A A A A A A B B B B B B c c c c c c DEF DEF DEF DEF DEF DEF GHI GHI GH I GHI GHI GHI JKL JKL J KL JKL JKL JKL ADM ADt1 ADt1 ADM ADM ADt1 BGN BGt·l BGN BGN BGN BGtJ CJO CJO CJO CJO CJO CJO Phase IV v VI Workshops to Questionnaire to Discuss Duties Rate KSAOs and Concurrent Region and Incidents Incidents Validity s t1 L s t1 L s t1 L 1 E H K ADPQ BGRS CJTU E H K 2 E H K ADPQ BGRS CJTU E H K 3 E H K ADPQ BGRS CJTU E H K 4 E H K ADPQ BGRS CJTU E H K 5 E H K ADPQ BGRS CJTU E H K 6 E H K ADPQ BGRS CJTU E H K Note: Departments symbo 1i zed as A,B, and c participated in Phases I I II I' and V; departments D ,G, and J 1n Phases II, I I I , and V; departments E,H, and K in Phase II, IV, and VI; departments F, I, and L in Phase II; departments M,N, and 0 in Phase II I; and departments P,Q,R,S,T, and U in Phase V. .tCJ' NORTHWEST NORTH CENTRAL WEST COAST SOUTHWEST SOUTHEAST Figure 8. Map of six geographical regions included i~ nationwide entry-level firefighter study. 47. A series of phone calls located the Municipal Yearbook published by the International City Management Association (ICMA, 1975). The yearbook which I isted I ,389 fire departments located in cities with populations greater than 10,000, was the result of a survey conducted by the International City Managers Association, but it did not include alI possible cities since the response rate of the survey was only 60%. We decided to selett the large cities by combining the ICMA I ist with the IAFC metropolitan 1ist. We also examined a geographical atlas to insure that the large city I ist was as complete as possible. We subdivided the 1ist of large cities into the six geographical regions and randomly selected cities to participate in different phases of the project. At this point, due to travel budget and time restri~tions, our plan deviated somewhat from a pure random sample. We randomly selected medium and small cities that were within a 200-mile radius of each targe city. In some instances, there were not enough medium cities available that met the geographical and size requirements. In these cases, we referred to the atlas and selected the medium cities that were closest to the large sample. We feel confident that the 42 large, 42 medium, and 42 small cities which we selected are representative of the total population of fire departments in the country because of the large sample size and thorough sampling plan. Of the 126 cities originally requested to participate in our study, 109 eventually performed in one or more of our five job analysis phases for an overall participa . tion rate of 87 percent. The 109 cities who worked with us in the course of the job analysis are 1isted in Append~x A. 48. V. IDENTIFYING FIREFIGHTER TASKS Definition of the Target Job. Our project focuses exclusively on the basic position of firefighter. Although it is cal led different things in different cities, we excluded from consideration positions which connoted apprentice, trainee, learner, or anything implying that an individual was not performing all tasks of a basic firefighter. This distinction is important because we encountered some confusion very early in the study when we referred to an entry-level firefighter and received responses related to an individual who is stil 1 in the train ing academy or on-the-job learning stages. We also excluded any promo tional positions such as equipment operator or engineer, I ieutenant, captain, etc. Thus, the job analysis was intended to uncover important tasks and duties performed by an individual who had completed prerequi site training prior to functioning as a firefighter. The clearest term we found for specifying the proper position was 11 private fire fighter with less than two years of experience. 11 Development of a Preliminary Task Checklist. Our initial job analysis chore was to construct a pre! iminary task check! ist to be used during visits to 18 fire departments. We began by reviewing the I iterature pertaining to the ftrefighter position. We reviewed approximately 18 published and unpublished reports, extracted task-related information from these reports, and recorded individual tasks and their source of reference in a standard format on 3x5 cards. The amount of editing required in this process depended on the type of material from which we extracted the task information. Material contained in critical incidents reported by various investigators required a considerable amount of editing. We filtered out information pertaining to the quality of task performance,and simply recorded task content. For example, 11 this fireman was nervousand forgot which way to turn the top of the hydrant to charge the hose ] ine11 was changed to 11 turn top of hydrant to charge hose! ine11 We • extracted a total of 1,093 task statements. PDI then assigned two staff members to review the task statements independently and identify approximately ten major categories or content areas. The two staff members then met and jointly agreed on the following categories: A. Maintaining bui !ding and grounds at the station. B. Inspecting and maintaining fire department equipment and apparatus. C. Rescuing people and applying first aid. D. Meeting the pub! ic. E. Analyzing situations and planning action. F. Investigating fires. G. Inspecting premises for fire hazards and for compliance with fire codes. H. Performing clerical and administrative tasks. I. Training self and others. J. Executing firefighting procedures. Two different staff members were asked to sort a sample of 63 statements representative of these duty categories into from 8 to 12 categories.Both sorters produced ten categories, and the overlap among these two sortings and the original categorization of 63 tasks was 87 percent. At a subsequent staff meet.ing we decided to combine dimensions E (Analyzing situations and planning action) and J (Executing firefightingprocedures) into a dimension called "Responding to fires". We then took task statements under each duty and sorted them into relatively homogeneoussubduties and, in some cases, into a further breakdown of the subduties. Since this version of the check! ist was tentative, we chose a very simple response format, with a "+'' indicating a task that was performed in a department, and a "O" if the task was never done •. Site Visits to 18 Departments. To ascertain whether our preliminary task analysis check! ist enumerated important activities performed by firefighters, we visited 18 departments across the country. Our site visits included one small, one medium, and orre large department frbm each of the six geographical regions. A staff member spent one day in each department interviewing experienced personnel and officials about jobactivities and tasks performed by firefighters. A copy of the materials for the visits is provided in Appendix B. Our initial interviews duringthe site visits with fire officials did not involve the direct use of our task checklist. Instead, we used a semistructured interview approach to elicit important information about firefighter tasks and duties. Later during the site visit, we asked firefighter incumbents and supervisors to complete. the task analysis checklist and to add any additional tasks which were not on our Jist. In addition to obtaining valuable information about the tasks in our check! ist, we also learned a great deal about the organization of fire departments and typical selection, training, and promotion procedures so. Although we did not find major differences in the basic tasks and activities engaged ·in by firefighters across our sample, we did find some slight variability. For example, departments which perform ambulance services for a city have somewhat broader rescue and first aid responsibilities than departments where private ambulance services exist. In most cases, the departments use a separate classification for engineers or equipment operators, but in a few departments these duties were part of the private firefighter's position. Some departments use civilians for dispatch and communications functions, while other departments use officer-level personnel, and a few use private firefighters. In general, both I ine and administrative personnel from alI 18 departments we visited agreed that our checklist was very comprehensive. Most recommended modifications were changes in wording or organization, and the few additional tasks our sample recommended were repeated in nearly every location. This gratifying outcome convinced us that no important tasks had been omitted from our Jist. We made several changes in the task analysis check! ist based on the site visits: our category 11 Responding to fires11 was changed to 11 Firefighting11; we improved the clarity of several task check! ist items by elaborating on their content; and we received valuable suggestions for the format of the check! ist from USCSC psychologists who used the list in the Washington, D.C. fire department. In addition, we used a very extensive job analysis recently completed in the Oklahoma City fire department, and the IFSTA Training Publications in revising our check1ist. Finally, two training officers and two battalion chiefs from the Minneapolis fire department reviewed our revised checklist and made suggestions for improving it. The result was ·a 204-item check! ist of the tasks performed by most of the departments we visited. 51. VI. PROCEDURES AND RESULTS OF TASK ANALYSIS CHECKLIST ADMINISTRATION PDI mailed the task analysis check! ist to 51 departments which had agreed to aprticipate in this project; we were unable to obtain cooperation from three midwestern cities that had originally been scheduled to participate. We received completed task analysis checklists from 48 of the 51 departm~nts. We asked departments to have a firefighter with at least two years of experience, and an immediate supervisor of firefighters (1 ieutenant or captain) each com~lete a ~hecklist. We also asked that additional checklists be completed by three other individuals in the department, such as the training director, department chief, or other personnel who were exceptionally well versed in the duties and responsibilities of firefighters. Fortunately, most of the departments returned five check! ists, and therefore, we were able to analyze data for 230 completed task analysis instruments. Needless to say, we were very gratified with this response rate and with the conscientiousness with which the check! ists were completed. A copy of the 204-item checklist and its instructions ~s provided in Appendix C. To assess the racial representativeness of the sample included in the task analYsis survey, we asked department officials to ihdicate the race of each person completing the checklist. Racial information was provided for 165 members of 35 departments. Of these, 12, or seven percent, represe~ted members of minority groups affected by Title VI I of the 1964 Civil Rights Act. For a future project phase, we asked 18 departments to indicate the numbers of their minority and white firefighters and received a total estimate of seven percent minorities. Furthermore, in a recent study Alvarez and Boston (1976) surveyed 70 cities across the U. S. finding that two-thirds had fewer than ten percent minorities and that only one-tenth had as high as 25 percent minority representation. P.uedebusch et al. (1975) report that of 97 firefighters participating in their multijurisdictional validation study, five, or five percent, belonged to minority groups identified in Title VI I. These results from several independent sources strongly suagest that our task analysis sample included minorities in approximately the same proportion as that in which they appear in the national firefighter population. Because our questionnaire was designed to elicit, for a relatively small number of raters from several levels of each department, their perceptions of the department 1 S firefighter job in general, is not appropriate to test for racial differences in job assignments by comparing the checklist responses of minorities against those of whites. No department which we contacted indicated having any female personnel in firefighting positions. Task checklist raters were first asked to indicate whether firefighters performed a task at all, and secondly whether they performed it on a regular or relief basis. We then asked them to indicate, using a 5 52. point scale ranging from 11 Smal 1 compared to others11 to 11 Large compared to others11 , the proportion of time they spent on each task relative to other tasks wl1ich they performed. We also asked them to rate the importance of each task using a 5-point scale ranging from 11 Less important than other tasks11 to 11Hore important than other tasks11 • 11 1mportance11 we defined as 11 the criticality of the task for achieving the avera! 1 objectives of firefighters which focus on preventing and extinguishing fires and saving 1ives11 • Although there were a large number of rating scales which could have been used for each task, such as frequency of performance, C01nplexity, or supervision required, we felt that the time spent and importance ratings would best enable us to identify the most preponderant and most critical parts of a firefighter 1 s job. We felt this was within the intent of the September 25, 1975, EEOCC Proposed Uniform ~uidel ines on Employee Selection Procedures, the latest draft of the Proposed Uniform Guide] ines at the time, which stated that validity studies should beginwith a job anlaysis to determine the important duties performed on the job, and should report t~e basis on which the duties were determined to be important or critical. For each task, we computed the meahs and standard deviations of import ance and time spent ratings and the proportions of departments which reported pe~forming the task on a relief basis only. Appendix D con tains these statistics for the total sample of 48 departments and for the six regional and three size subgroups. Tables 6 and 7 show the inter region and inter-size correlations for the importance and time spent ratings. These correlations in the middle and upper .80 1 s for regions,and in the middle .90 1 s for sizes, indicate excellent agreement across regions and across sizes for the task analysis ratings. The lowest standard deviation for importance ratings was .34 for item 70, 11 Don and operate breathing equipment11 • This item also had an extremelyhigh mean rating (4.6 on a 5-point scale). The highest importancerating standard deviation was 1.31 for item 129, 11 Provide paramedical treatment to victims in apparatus while enroute to hospital, using UHF transmitter, and following treatment advise<;! by doctor on call 11 • Obviously, there was considerable variability among departments concerning the importance of this item and probably about whether it was performed at all. The lowest time spent standard deviation was .08 for item 33, 11 0perate fire/rescue boat at scene of water front fires 11 • The highest standard deviation was 1.13 for item 129 described above. The basic purpose of analyzing the importance and time spent ratings was to identify the most important firefighter tasks so that theycould be further studied in subsequent project steps. We did not want to eliminate potentially important tasks nor did we want to work with 53. an overly cumbersome list of firefighter duties. Since some tasks are highly important but infrequently performed while others are performed quite often but are not overly critical, we gave equal weight to the two ratings and computed the cross products of the time spent and the importance ratings. This formed a compensatory index since an extremely high rating in one scale cou1d compensate for a lower score in the other. The cross product of the mean department time spent and importance ratings was computed for each task across the total sample of 48 departments and within each of three department sizes and each of six geographical reg ions. We also computed the proportion of departments in which firefighters performed each task on a relief basis only. Tasks with low time spent importance cross product means and high relief only proportions were eliminated from subsequent project phases. We retained 143 tasks having at least a 5.0 cross product rating, implying that ai least one scale's rating could have reached a 5.0 or very important value. We inspected individually the 143 items retained and eliminated 25 tasks which had both time spent and importance ratings of approximately 2.25. Next we added two tasks with time spent ratings of approximately 1.0, but mean importance ratings greater than 4.0. Finally, to ensure that no tasks of importance had been eliminated, we computed the average rating variance within departments for the cross product index for each task. High within department error variance would indicate dis agreement.among raters within departments about the importance of a task, and would imply that at least some raters thought the task of high importance, despite a low mean rating. We found very good inter. rater agreement on the task i terns with low average importance, however, increasing our confidence that no important tasks had been eliminated. At this stage; the 120 most important tasks from our original 204 remained. 54. TABLE 6 REGIONAL INTERCORRELATIONS IMPORTANCE AND TIME SPENT FOR TASK RATINGS Regions ]· 2 '87''•• Regions 3 88 4· 86 ' '"5' 85 6 88 2 87 87 94 87 90 3 88 87 89 86 88 4 83 92 88 90 89 5 81 87 84 86 90 6 ' 86 88 . 83 83 89 Note: Correlations for importance ratings are above the diagonal. spent correlations are below the diagonal. Decimals omitted clarity. Time for 55. TABLE 7 SIZE ,INTERCORRELATIONS FOR TASK IMPORTANCE AND TIME SPENT RATINGS Size s M L Size s 96 93 M 94 95 L 93 94 Note: Correlations for importance ratings are above the diagonal. Time spent correlations are below the diagonal. Decimals omitted for clarity. 56. VI I. CLUSTERING TASKS Rationale for Clustering. Our .next step involved categorizing these tasks into a smaller, more manageable number of task clusters. Subsequent project phases cal led for generating job related information about firefighter tasks and duties, including their difficulty level, their prerequisite KSA 1 s, and several behavioral performance examples. Categorizing tasks into a smaller number of dimensions or duties would reduce these problems to manageable proportions. Typically in studies similar to ours, this col lapsing has been accomplished by researchers who classify tasks into content categories. However, researchers are not always sufficiently familiar with the firefighter 1 s job and their judgments may not always be accurate. A less frequently used approach has been to factor or cluster analyze either time spent or importance ratings for tasks, and to employ the resulting task groupIngs. The danger with this approach is that time spent or importance clusters could emerge which have no meaningful relationship to the way firefighters perform their job, or to individual differences among firefighters. Similarity Rating Task. O~r approach to clustering tasks was'to obtain similarity ratings between all pairs of tasks in our 120-item task check1ist. To accomplish this, PDI conducted seven workshops across the country (Detroit, Syracuse, Atlanta, Omaha, Denver, Seattle, and Los Angeles) attended by 93 participants from 46 departments. Typically, a department sent a firefighter with two to eight years of experience, and a company officer who supervised firefighters. We pilot tested a method for directly estimating task similarities in Minneapolis, by asking firefighters simply to rate how similar two tasks were, based on how wel 1 people they knew performed each task. The Minneapolis participants found this rating process to be extremely difficult, however, and the method was revised. In a second pilot test in St. Paul, we presented a technique essentially identical to our final procedure with excel lent results. The St. Paul participants had little difficulty perform-ing the task, and only minor changes in the instructions were required. The materials that emerged from this second pi lot test are presented in Appendix E. We asked each participant to rate the similarity of 12 target tasks to each of th~ remaining 119 tasks. The target tasks were systematically assigned to raters in a way which ensured that performance on each task was compared with performance on every other task by a maximum number of raters. The 12 targets each described a firefighter who was either 57. above average or below average on one task. Raters were asked to decide, given only that single piece of information, how they expected the target firefighter to perform on each of the remaining 119 tasks. Each rater received six above average and six below average targets, and the target task assignment was designed such that with 100 raters, every task was compared with every other task exactly ten times, five as an above average target, and five as a below average target. Participants rated the expected performance of the target firefighter on a 5-pointscale, ranging from -2 (Definitely below average), to +2 (Definitelyabove average), 1-vith zero representing no predictabi'J ity from the target task to the task being rated. Rei iabil ity and Cluster Analyses of Similarity Ratings. We obtained an average of between eight and nine raters for each item pair. To assess the inter-rater agreement on similarity ratings we computed, for each item, the Hoyt rei iabi I ity for the ratings ~cross 119 remaining items. First, these reliabilities were computed separately for similarity ratings made on above average and on below average target tasks, then the below average ratings were reflected (multiplied by -1), combined with the above average ratings, and the Hoyt (1941) rel iabi I ity for these combined ratings was computed for each. item. Table 8 shows the above average, below average, and combined rating rel iabil ities for the 120 tasks. The majority of the combined rating rel iabil ities exceed .80, and only eight tasks had rei iabi 1it ies below .SO, indicating generally excellent inter-rater agreement on task similiarities. The mean similarity ratings among tasks for the total sample are shown in Appendix F. To identify the major dimensions underlying expected performance on the 120 firefighter tasks, we performed cluster analyses of the tasks using the techniques described by Ward and Hook (1963). For the first analysis, the similarity rating values for each rater were rescaled byfirst reflecti~g al 1 ratings for below averagi target tasks, and then arbitrari Jy converting alI ratings of 1.0 to .5, and al 1 ratings of 2.0 to 1.0. These rescaled values were averaged across raters, to produce a set of data approximating an intercorrelation matrix for the 120 tasks. This matrix was factor analyzed, all positive roots extracted, and the "correlation" matrix reconstructed from these factors. Although the residual matrix, after the positive roots were extracted, was very small, this procedure ensured that no logical inconsistencies existed in the relationships composing the reconstructed matrix. This reconstructed "pseudocorrelation" matrix was subjected to a hierarchical cluster analysis, which sequentially identified the highest correlating pair of ~he 120 variables, averaged this most simi Jar pair and replaced them as a single new variable, and scanned the new 119-variable matrix for its highest value, continuing until al 1 task items had been collapsedinto-a-single cluster. Figure 9 shows an example of how this clustering process WGrked for ten of the items in our similarity check! ist. The 58. TABLE 8 HOYT RELIABILITIES FOR RATINGS OF EXPECTED PERFORMANC~ SIMILARITY BETWEEN TASKS .::L IJ'l cu I 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 CJa 3b ~c 0., 0 ~ (!) cu -1-J o· .::L 'Jl cu > 0 ...0 0 -(!) ~ a:l I- I 0 <0 ..:,{. ~C Vl 0 ~ ..... > ~ 0 <0 Ql Vl 0 ..... <0 ..0 0 <0 ..0 Ql 01-<:{ c::l 1-1-<:{ co 1 61 64 59 77 91 68 72 82 62 76 74 86 92 79 72 8763 54 58 73 93 64 83 86 64 52 41 64 94 37 54 6365 19 13 27 95 47 71 74 66 49 60 70 96 79 81 8967 61 71 80 97 56 51 71 68 61 64 77 98 26 29 4369 38 02 35 99 66 66 8070 76 68 84 100 I 7 36 4271 63 64 78 IOJ 47 52 6672 81 64 86 102 77 76 8773 77 66 84 103 53 46 6674 67 71 82 104 54 53 6975 82 81 90 105 53 63 7376 68 74 83 106 77 76 8777 27 30 44 107 70 73 83 78. 74 60 81 108 71 77 85 79 87 90 94 109 73 69 8380 68 73 83 110 7-4 68 84.81 77 90 91 I I I 87 87 9382 76 87 90 112 46 64 7183 70 80 87 113 15 53 5284 83 86 92 114 38 56 6585 82 87 92 115 25 41 5286 70 80 86 116 8o 67 8687 68 76 85 117 64 68 7988 74 79 86 118 39 39 5689 64 70 81 119 64 74 8290 75 86 90 120 60 59 75 Note: Decimals omitted for clarity. a. Rei i.3bi 1i ty of expected performance ratings for above averagetargets. b. ReI i ab i"l i ty of expected performance ratings for below averagetargets. c. Reliabi I ity of rating values computed by reflecting belowaverage target ratings and combining with above averagetarget ratings. 60. 75 86 78 87 70 71 73 74 (1) Task items 52. Coi 1 and throw rope and tie knots. 70. Move heavy objects or material~ to gain access to or to fre~ trapped victims. 71. Locate and dig to free victims trapped or unconscious in tunnels, pipes, sewers, etc. 73. Carry conscious, unconscious, or deceased victims down ladder or stairs using drags, slings, cots, scoops, chairs, stretchers, or improvised equipment. 74. Remove victims using 1ife gun, lines, and belts. 75. Hoist and lower victims using rope and tying knots. 77. Remove energized electrical wire from contact with victim u~ing hot 1ine cutters, or pike pole, and protective clothing. 78. Carry victims to ambulance or other emergency vehicles to assist emergency medical personnel. 86. Rescue drowning persons using poles, ropes, buoys, boats. 87. Rescue drowning persons by swimming and using appropriate 1ifesaving carries. Figure 9. Example of clustering sequence for ten rescue task~. 61. ten individual tasks represent the first level of the analysis. The two most similar tasks, 73 and 74, were clustered first, followed at the third level by tasks 70 and 71. At the eighth level, these two clusters were grouped together, and at the tenth level, al 1 ten tasks have been grouped into a single cluster. Groups formed at the earliest levels are most internally consistent, but typically consist of so few items that the total number of clusters at these levels is excessive. In the second analysis, we computed, for each pair of tasks, the correlation between their ratings across the remaining 118 tasks, and subjected the resulting correlation matrix to a hierarchical cluster analysis. We studied the results of the two clustering methods, examining the task groups produced at each clustering level, and we identified the most 11 mean_ingful 11 solution for each method, based on the content of co-clustered tasks, and the ratio of the average within-cluster to the average between-cluster correlations. Since the two solutions were common not identical, our final clusters comprised first, the results to both, and then our interpretation of the most meaningful combination where the solutions differed. We eventually identified 15 clusters, which we tentatively named: 1. Rescue 2. Salvage and overhaul 3. Ladder 4. Apparatus operator 5. Dealing and interacting with people 6. Forcible entry ]. Preventive maintenance tasks for apparatus and equipment 8. Hose evolutions and extinguishing fires 9. Inspection and pre-planning 10. Emergency care 11. Physical strength 12. Evaluating fire conditions 62. 13. Routine in-station duties 14. Communications 15. Training and educational activities All tasks in each category or cluster had been rated as similar, in that firefighters who performed wei I on one task in a cluster would be expected to perform well on all tasks in the same cluster, but not necessarilyon tasks in other clusters. For example, a firefighter above averageon tasks involving operating fire apparatus would not necessarily beexpected to perform well at various first aid functions. Overall, wefelt we were successful in identifying a set of meaningful and homo geneous clusters of tasks which reflected the individual differences under lying performance as a firefighter. VIll. REGIONAL WORKSHOPS TO GENERATE INFORMATION ABOUT TASK CLUSTERS To gather additional information a,bout the 15 ta~k.. clusters derived from our task similarity analysis, and to pretest the materials to be used in a subsequent national survey, we conducted four two-day workshops in Philadelphia, Minneapolis~ San Francisco, and Houston. Of 18 departments invited to participate in these confererces (one small, medium, and large from each regionl, 16 sent a total of Jl representatives, mostly department training directors, battql ion chiefs, or administrative officials,~ith extensive 1ine and supervisory experience. The 31 participants averaged 21 years of fire service experience, with eight years as firefighters and seven years as company officers. Training and administrative personnel averaged six years of experience in these positions, and the total sample averaged four years experience as department-level training officers, and three years experience as administrative officials at the battalion level or hiqher. We asked these workshop participants first to review and discuss the 15 clusters we had obtained from our clustering process. They were encouraged to modify the clusters, if necessary, to insure that cluster content was meaningful and that similar characteristics were contributing to performing all the tasks in each cluster. In the first workshop, the cluster 11 Physical strength requirementS11 was dropped since phys.ical strength seemed to cut across most other dimensions, and 11 Applying ventilation procedures11 and 11 Following safety practices11 were added as extremely important clusters, which had previously been omitted. When we asked workshop participants to discuss the relative importance of the 16 resulting task clusters, two conclusions emerged: that rescue operations were relatively more important, and that further discussions of relative importance were meaningless since none of the remaining clusters could be omitted without seriously jeopardizing fire service goals. In addition to rating task similarities, participants in our prior workshop had been asked to consider the 36 duty headings into which our tasks were organized at that time, and to list the knowledges, skills, abilities, personal qualities, and background characteristics which firefighters should have to perform the tasks included under each duty. Beginning with this information we developed a comprehensive 1ist of 62 abilities and characteristics that appeared to be essential for successful firefighter per.fonnance. In developinQ this list we drew heavily from McCormick 1 s work on the human attributes required for performing PAQ 64. elements, paying particular attention to those elements which had been judged to be important for the firefighter's job in other studies. We also carefully considered the Selection Consulting Center's Jist of attributes, AIR's work in New York City, and the results of our own studies in three departments. The preliminary Jist of attributes is presented in Appendix G. Workshop participants discussed the 62 attributes on our I ist, andconcluded that it was comprehensive, although some of 1the definitionswere excessively abstract. Some participants felt the list was too long,largely because of redundant items. We asked representatives to make two kinds of ratings for the 62 attributes. First, they estimated how much of the characteristic was trainedor developed after hire, and how much was required at the time a recruitentered the fire service. Next, participants rated the importance ofeach attribute for performing adequately in each of the 16 task clusters.In two workshops, participants were asked to rate attributes' importancefor both adequate and superior performance. No differential patterns ofrequirement ratings were found between the two sets of ratings, andsup~rior ratings were typically one point higher than adequate ratingson a 5-point importance scale. Fifteen raters part1c1pating in two of our workshops made ratingsfor all 62 attributes across the 16 task dimensions. We computed theaverage importance retina for each cell in the 16-by-62 matrix, anddivided the attributes into three categories: mental and perceptualabi I ities, physical abilities, and temperament characteristics. Foreach of these three categories, we computed the correlation betweeneach pair of attributes for. the profile of importance ratings acrossthe 16 dimensions, and factor analyzed the resulting correlation matrixusing a principal factors technique and varimax rotation. Twenty mentaland perceptual abilities produced four interpretable factors which wedefined as visual acuity, formal skills (verbal comprehension, writin-gability, and math skills), problem solving ability, and mechanical,abi I ity. The physical ski lis produced four factors, including physic~!1•strenqth, general body coordination, quickness, and arm-hand dexterity.The 26 temperament characteristics also produced four factors: emotionalcontrol (courage and resistance to stress were the two major componentsof this factor), responsibility, interpersonal skills, and interest inthe construction trades and related activities. These factors were laterused to help reduce the number of personal attributes considered in sub-·sequent pr~ject phases. Tables 9-11 show the factor matrices for theseanalysis. We also performed the inverse analysis by correlating pairs of task clustersacross their importance ratinQS for the 62 attributes, and factor analyzinathe dimension intercorrelation matrix. The best solution produced five 65. TABLE 9 FACTOR LOADING MATRIX FOR THREE FACTORS AND 15 PHYSICAL ABILITIES Factors . , , .. Abi1itya II Ill 12 -71 -62 -24 • 14 -35 -41 -83 15 -60 -37 -70 16 -60 -47 -63 17 -70 -42 -57 18 -75 -44 -45 19 -60 -ss -54 20 -77 -42 ,..39 21 -35 -82 -'43 22 -46 -76 -41 23 -83 -38 -38 24 -83 -36 -40 25 -85 -39 -34 27 -83 -31 -40 28 -67 -59 -40 Note: Decimals omitted for clarity. a. See. Appendix G for ability definitions. 66. TABLE 10 FACTOR LOADING MATRIX FOR FOUR FACTORS AND 20 PERCEPTUAL AND COGNITIVE ABILITIES Factors Ab i 1 it iesa II II ,I IV 11 -96 08 -16 2 -02 -92 17 • -24 3 53 -75 -16 15 4 50 -69 38 -12 5 69 -15 65 14 6 67 -:-08 70 05 7 80 -13 47 21 8 88 -33 30 07 9 89 -17 34 -OJ 10 87 12 43 15 11 49 21 70 24 13 65 49 46 11 26 55 34 26 69 35 21 -04 95 06 37 26 -33 84 19 38 60 32 69 04 39 43 -01 86 -14 so -05 -98 04 04 51 -11 -97 -01 06 53 31 -51 78 -03 Note: Decimals omitted for clarity a. See Appendix G for ability definitions. 67. TABLE 11 I FACTOR LOADING MATRIX FOR .SIX FACTORS AND 26 PERSONAL ATTRIBUTES Factors Attributesa I II Ill IV v VI 29 -67 -24 38 -19 54 -09 30 -53 -31 42 24 51 -04 31 22 -32 -85 -10 -16 -04 32 -92 -10 34 13 01 09 33 -64 -58 39 -18 -07 -04 35 -49 -77 18 20 09 02 36 -84 -10 42 07 26 -07 40 -89 -03 30 25 19 -01 42 46 27 -79 -09 -06 21 43 44 . -13 -79· 20 03 -19 44 -37 -75 -23 22 01 11 45 45 22 -76 -09 -02 21 46 -92 -18 27 10 09 06 47 33 05 -86 -30 -08 -Ol 48 -71 -54 36 -as 05 ' -12 49 -77 -65 20 05 56 -11 52 10 -94 -14 -06 07 03 53 -23 -95 -02 08 -05 12 54 05 -43 -57 -19 -11 64 55 46 05 -83 02 -06 22 56 13 -44 -83 11 -15 06 57 22 -76 -23 -22 25 -13 58 -52 -so 09 33 42 34 60 -47 -43 -65 -23 -02 -17 61 -27 -04 13 94 03 -06 62 -89 02 12 21 -08 -01 Note: Decimals omitted for clarity a. See Appendix G for attribute definitions. 68. factors: fire response activities, public contact, routine activities, operating apparatus, and administering emergency care. The task cluster Performing hose evolutions had a unique split loading across factors one and three, and two other dimensions, Fire knowledge and PerforMing preplanning inspections, each also showed a unique pattern of split loadings across the five factors. The factor matrix for performance dimensions is presented in Table 12. We tried several methods to obtain difficulty ratinqs for the task clusters. We were interested in finding out whether the clusters were so difficult and demanding that only the most experienced, highly trained, and competent firefighters could perform them or whether they were so easy that nearly any firefighter could perform them adequtely without training. Although department representatives had some trouble rating the difficulty of entire clusters, since the tasks within clusters varied greatly in complexity and difficulty, we did have some success in estimating the number of days required to train an individual to perform all of the tasks in a cluster competently. Another major task performed durinq these workshops was writinq incidents or behavioral examples representing inadequate, fully adeouate, and superior performance by firefighters on each of the 16 task clusters. These incidents were written so that we could develop performance ratingscales which would provide objective behavioral standards for evaluating the performance of a sample of current firefighters. 1.-lorkshop participants generated more than a thousand 1ncidents. PDf edited the incidents and eliminated redundancies, producin~ a total of 723 usable critical incidents: Finally, we asked workshop participants to describe the conditions under which the tasks in each cluster are typically performed. Of special concern here were the circumstances that make a duty relatively more difficult on some occasiQns and less difficult on others. Appendix H provides a summary of the conditions or circumstances reported to influence performance. 69. TABLE 12 FACTOR LOADING MATRIX FOR FIVE FACTORS AND 16 TASK CLUSTERS Factors II Ill IV v Clusters 18 -03 -31 1. Rescue -90 17 -08 2. Salvage -90 30 19 -07 -04 26 08 3. Ladders -90 20 06 -02 98 02 4. Apparatus -09 5. People 14 -92 28 -16 03 06 05 -23 6. Entry -87 37 -30 7. Maintenance -39 00 84 07 33 03 8. Extinguishing -70 10 59 01 9. Inspection 35 -93 -04 03 10. First Aid -38 -09 58 -07 -69 26 27 -03. -04 11. Venti 1at ion -89 09 -12 12. r ire Knowledge 26 -84 -43 94 -08 -06 13. Routine -28 15 14. Communications 29 -75 56 -10 -09 15. Training 23 -68 60 -12 15 18 04 -01 16. Safety -90 09 Note: Decimal omitted for clarity. 70. IX. NATIONvJIDE SURVEY QUESTIONNAIRES TOGENERATE INFORMATION ABOUT TASK CLUSTERS Rationale and Questionnaire Development. In this phase, we mailed questionnaires to a large nationally representative sample of fire departments to obtain reliable information regarding firefighter per formance dimensions, requisite ski lis, and performance examples. To reduce the number of human attributes to be considered in this sur vey, we evaluated the factor analyses of the 62 attributes from the previous project phase. Since our major emphasis is on qualification areas other than cognitive and physical ski! Is, we employed the factors from these two attribute categories relatively unaltered. We did, however, split math ski! Is apart from the more general formal ski 1Is factor.We collapsed the 26 temperament characteristics into 11 attributesby combinLng only those which loaded most highly on each of the tourtemperament factors and whi~h appeared to share a good deal of commonmeaning. Characteristics which loaded relatively low on al 1 fourfactors and those with split loadings were split out as separate attri butes for consideration in.the survey. At this point, we also added a task dimension. The literature wereviewed in Section I I consistently reported a performance dimen~ionrelated to maintaining interpersonal relationships with other firefighters, and informal ·estimates by the departments we visited in ourfirst phase indicated that from one-half to two-thirds of a firefighter'stime may be spent without assigned activities in the company of thestation crew. We had previous!~ incorporated this aspect of a fire fighter's job in the Station routine cluster, but felt that this strategy gave inadequate emphasis to interpersonal skills. For this reason,we added a 17th task cluster titled Getting along with peers, and weadded critical incidents collected in previous studies POl had performed, to ensure that a sufficient number of behavior examples for this dimension were included in our survey. Appendix I contains definitions of the 20 abilities and characteristics and of the 17 task clusters. Since four of our initial 72 departments indicated they could no longer participate in our study, we mailed survey materials to a total of 68jurisdictions. After a series of follow-up telephone calls, 53 departments, or 78 percent, returned completed questionnaire packages, anexcellent response rate given the magnitude and complexity of the rating task involve.d. Our original sample design had called for four departments in each of 18 size-by-region eel Is. The 53 returns includeddepartments from al 1 eel Is, although two eel Is were represented byonly one department (a medium-sized Southeast and a smal 1 Midwest depart 71. ment). In general, we found that large departments and departments which had been involved in earlier phases of our study were more conscientious in completing the questionnaires than were other jurisdictions. Within departments, returns were higher for those rating tasks assigned directly to ou~ liaison person in the d~partment than for those which the liaison delegated to other de~artment personnel. We had several critical objectives for this phase of our project. W~ wished to determine the relative importance of the 17 task dimensions and the level of difficulty at which tasks in each dimension were per formed. We wanted to distinguish skills required at the time of hire from those to be trained after selection. We desired to learn how various skills contributed to performance across dimensions, and we wished to have the performance examples which had been written in our earlier workshops sorted into dimensions and rated for their effective ness. Questionnaire Sample. The magnitude of this rating task prompted us to suggest ways for dividing up the workload within each department. The letter to the department 1iaison officer, which accompanied the questionnaire materials, is shown in Appendix J. In this letter, we asked departments to assign a direct supervisor of firefighters to retranslate the behavioral examples. We asked that the chief training officer rate the training time required for each task (to estimate task difficulty) and that he and one other official estimate initial ability requirements. We asked that the liaison officer or another department official indicate the importance of our 20 personal characteristics for performing each of the 17 fir~fighter duty clusters. Finally, we requested that a firefighter with five years of experience, a company officer, and a chief or department level official alI rate the rela tive importance of the task clusters. We believe that splitting and distributing the workload in this manner was largely responsible for the relatively high return rate we obtained for such an arduous rating task. Dimension Importance Ratings. We desired to assess dimension importance, first to ensure that all our task clusters represented crucial, nontrivial aspects of the fireflghter•s job, and second to develop relative importance indices to weight the predictor composites selected to estimate perfor111.1nce on the final dimensions. In obtaininq these importance ratings, we asked participants to consider both how critical or crucial the dimension was for achieving overell firefighter objectives and how frequently the tasks in the dimension were performed. Tornow and Mabey (1975) used such a combination importance index in a task analysis of execu 72. tive positions, and we adapted their rating scales and instructions to our study. We asked three respondents from each department to rate each dimension's importance on a 6-point scale, ranging from 0 (11 Not per formed by firefighters in this department11 ) to 5 (11 Definitely a most significant part of the firefighter's job11 ). The form used for this rating is shown in Appendix K. A total of 131 fire service personnel from 49 departments completed the task dimension importance ratings. Respondents )ncluded 17 experi enced firefighters, 47 company officers, 45 battalion, district, or assistant chiefs, and 14 department chiefs. Table 13 shows the rank ordered rating means and the rating standard deviations for the 17 task dimensions includ~d in our questionnaire. The means show that all dimensions except M, Performing communications duties, were rated as above median importance. Thirteen dimensions received rating means greater than 3.5, indicating that these perform ance areas are ~ajtir aspects of the firefighter's job. Since these statistics are based only on data from departments in which firefighters regularly perform the tasks, the table also shows the proportion of departments in which firefighters do perform each task cluster as part of their assigned duties. These values, except for the communications cluster, are quite high indicating that firefighters are normallyexpected to perform all the duties represented by our clusters. The rel inbil ity of these dimension importance ratings was assessed several ways. First, using the Hoyt formula, the reliability of the importance rating profile across 17 dimensions for all raters was .97.This value shovJS that the mean ratings presented in Table 13 are.extremely reliable indicators of the relative importance of the 17dimensions. The means could also appropriately define a criterion importance profile with which importance ratings from new departmentscould be compared to assess the similarity of firefighter functions inthe new to those of the development sample departments. A Spearman Brown correction to the total sample rel iabi 1ity suggests that ideallyat least ten dimension importance raters should be obtained from eachsuch new department. This many raters would ensure that the department's mean importance profile is sufficiently reliable (i.e. interrater reliability approaching .80) to justify a conclusion that theprofile is either similar or dissimilar to that of the national sample. The extent of agreement across regions and across sizes of departments is indicated in Table 14, which shows the correlations across the 17dimension importance ratings for all possible pairs of regions andof department sizes. For various sized departments, these correlationsare uniformly high; the values for regions show somewhat greater variance. 73. TABLE 13 TASK DIMENSION IMPORTANCE MEANS AND STANDARD DEVIATIONS AND PERCENT OF DEPARTMENTS GIVING NON-ZERO RATINGS Percent depts. with all non.:. Dimension t~ean Sd zero ratings H. Performing hose evolutions and 4.42 .]2 100 applying extinguishing agents F. Following standard safety 4.38 .n 100 procedures Q. Getting along with peers 4.35 ;73 100 A. Performing rescue operations 4.22 1. 10 100 N Participating in training and 4.07 -79 100 education B. Performing salvage and overhaul 3-93 . 84 100 E. Applying ventilation procedures 3.86 .94 100 C. Performing ladder operations 3.86 .96 100 P. Performing routine station 3.70 1. 03 100 duties D. Forcibly opening structures 3.64 1. 06 100 and enclosures G. Applying knowledge of fire 3.62 1. 12 98 characteristics J. Administering emergency care 3. 61 1. 15 100 L. Performing preplanning and fire 3.42 1. 04 98 prevention inspections 0. Reconditioning and maintaining 3-39 1. 17 94 equipment K. Dealing with the public 3-37 1. 05 100 I. Operating apparatus 3.36 1. 23 92 81 M. Performing communications duties 2.48 1. 13 74. TABLE 14 REGIONAL AND SIZE INTERCORRELATIONS FORTASK DIMENSION IMPORTANCE RATINGS Region 2 3 4 5 2 60 3 86 57 4 69 67 85 5 56 75 64 83 6 71 67 83 91 73 Size s M M 90 L 82 93 Note: Decimals omitted for clarity. 75. In particular, the agreement between Northeast (Region 1), and West Coast (Region 5) raters is fairly low (r=.56), as is that between the Southeast (Region 2) and both the Midwest (Region 3) and the Northeast (Region 1). All the values in the matrix are high, however, and one should not overlook this high communality in examining variations within the matrix. These trends are confirmed in a Multivariate Analysis of Variance (MANOVA) which we performed to identify significant regional and size variations across the 17 dimensions. MANOVAs test for dependent variate mean differences across the eel ls of the independent variables (in this tase, six region cells, three size cells, and 18 region~by-size cells) where the dependent variate is a composite of individual dependent variables (importance ratings for our 17 dimensions). This analysis showed a significant effect for regions using Bartlett's (1947) tPst of siqnific~ncP, but no~e for department size or for the region-by-size interaction. Between regions, univariat~ F-tests on the individual dimensions were significant at the .05 level for 9 of the 17 dimensions: A, C, D, E, I , L, M, P, and Q. Of these nine dimensions having significant regional mean differences, four were interpreted as unimportant. Dimension C, Performing ladder operations, received mean ratings within each region of at least 3.5, indicating al 1 regions considered the dim~nsion to be a major part of the job. In addition, the region effect for Dimension C was attributable to a chronic tendency for Northeast raters to use the top end of the scale and for West Coast raters to use the lower end as indicated by their mean ratings over all 17 dimensions. The significant region effect for Dimension E, Applying ventilation procedures, was produced by a low rating from our only medium-sized Southeast department (Southeast region importance rating mean= 2.33). This department had extremely low intradepartmental rel iabi 1ity, and given the high importance ratings obtained from other" departments (mean=3.89) and the "face valid" importance of the dimension, we felt justified in overlooking this region effect. \~e anticipated low interdepartmental agreement for Dirncnsir.. I Operating apparatus, and we shall deal with this problem in suh··'qrJ·:nt phases of the study. The overall importance rating for Dimension M, Performing communications duties, was so low that we intend to delete this task cluster as a trivial portion of an entry-level firefighter's duties. Finally, the regional means for Dimension Q, Getting along with peers, are all extremely high and show relatively 1ittle variance (range= 3.9 to 4.7). Dimension A (Performing rescue operations) and Dimension D (Forcibly opening structures) also produced significant region effects which initially concerned us. These effects were, however, due solely to an 76. extremely low rating given by three smal 1 West Coast departments. (Dimen sion A mean= 1.5 for one department; Dimension D mean = 3.1 for al 1 three departments). Follow-up conversations with training and administrative personnel from these departments indicated that the original ratings were erroneously low and that the ratings For Dimension A should be at least 4.0, and that those for Difflension D should fa! 1 between 3.5 and 4.0 closely approximating the results for the balance of the departments in the sample. Dimension L, Performing preplanning and fire prevention inspections, like Dimension I, appears to be performed inconsistently across regions. Midwest, Southwest, and West Coa~t departments all gave relatively low importance ratings for this performance area (the mean for these three regions was 3.1 I). Because of this inconsistency, we shal 1 develop both performance evaluation measures and selection instruments for this dimension; but we shal 1 provide local jurisdictions the option of eliminating these measures from a predictor com~osite if their fire fighters are not expected to perform preplanning and inspection duties. The remaining dimension which produced a significant region effect, Dimension P, Performing routine station duties, is somewhat perplexing. Of seven Midwest departments, including all sizes, six rated Dimension P substantially lower than the mean for the remaining departments (2.94 vs. 3.81). It is n I ON AIHL A B c D [ F G H I J• I·( L t1 N () ..1··, Q 1 2 4.0 3.8 3.6 3.4 3.8 4.1 3.9 3.? 3 y ~5 3. ~.'i 3. l 3.3 ..") ·=· ...:. + .... J '1 ..., ,:_ + I ~3. (? 3.7 2.B 3.B 2.8 :L ~.:; ~~. 4 ..., -, ...·:. + I 2.4 '") i ...·•. + \.) 1 C) . t ... r) "".' ..._ + / 3.1 ~5. 2 2.7 2.8 '1 '1 ..... + A2.4 2.1 ,., ll A.. -y 3 3 + ~i 3.1 A r.:·'\.+ ..J 3.6 3.2 3.3 3.1 :L5 3.7 3.7 2.6 2.4 3. ~~ 3.:1. 2.6 ::~ • A ,., '7 ..·•. + \J 4 5 ~·L4 3.4 '7 -) V+~ 3.2 3.6 3.4 3. t1 3.4 :~ + ::.~ 3.1 3.4 3.2 -> .(...._ + \..) 3.8 :3 + "/ 3.:1. 3,-7:1 + ~v' 3.9 3.4 :.>. 7 r) 9 ...•• < 2.7 3.7 3.0 3. :1. 3. 1 3.3 '1 0 ~~ ) 3.2 ::.~ + ~:.) '") ' ,..·_ + \~ 2.3 '} 1::· ...:.. .. '-·' 6 -,I 3.0 2 + 1 2.7 2.0 2.? '1 fl ~·.• + y '1 .., ,,._ <-I '") '")•.:.. + ~.:.• 2 + ~.:; 2.0 2.5 :;~. 0 3.3 3.0 2.7 2.3 3.0 :~ • B 3.6 '"> .i.""'·· • ~J 4.:1. :~ + 4 4.0 3.1 4.2 '") '") ..••• .00 .,·•• 3.6 3.3 :.~. 6 2.4 '") 1::· ...•.• + \-' 1.9 3.:5 2.4 B 9 lO 3.6 3. ~j 4 '') . • A.. 3.3 3.1 2.9 :-z; ,, "-· + ...... 3.2 3.7 3 "7 ' + / 4.0 3.4 3.0 3.3 ~3. 2 3.0 3. :1. 3.1 3.9 3.0 3.:1. 3.1 3. ::~ 3.:1. 3.7 4.0 2.7 3.8 2 + Cj 3.2 2.9 .-) _.( ,;•• <~.) .., t::~to".: t ,_. 3.6 3.5 2.4 ~5.2 '") ··r 1>:•• (> \ ••• 2.1 3. :=.=; 3. :::~ 2.? '") -·} ,,;... / 3.4 2.1 ") '') ..·.. + ,;•. '") '")..·....,·.. 1..9 3.0 ~-~. 6 2.4 1.1 l2 3.9 4.0 2.9 3.8 3.6 4.2 3. !'.'j 3.9 3.4 ~5. 7 3.3 •oy ··-~. ,'j 3.7 3. !:'j 3.4 4.1 3.7 :L ~5 3.9 4.0 :~~ + 6 :2.9 2.6 3.0 3.6 3.4 ,., C) A+ .I 3.6 ::.~. 4 "1 ,., ...J ~ ..•.. ;~ + l 3.4 3.0 4.0 13 14 3.6 3.? 3.5 3.4 3.4 3.4 3. f.· 3.6 3.4 3.2 3. :1. 3.4 3.3 3.9 3./ 3.4 3.3 3.9 3. ~'5 4. 1 3.1 3.6 3.0 3.7 3.0 4.0 7 ~ ~•0 3.5 3. :.~ 3.7 3. 1 3.4 3.3 .., ,-· ~. ,'j 15 16 l7 3.6 3.4 3.5 3.4 3.2 4.4 3.4 3 '')....._ 3.0 3.6 3.3 ....., ...).;) + ,,,. 3.3 3 • .1. 3.0 3.5 3.0 3.1 3.7 3.0 3.1 3 1::" + ...) 3.2 3.0 3.8 3.1 3.0 4.0 3.6 3.4 3.4 3.9 3 + C,)l 3.8 3.? 4.0 3.7 3. ::) ..... ,~...") + .:} 4.1 3.7 3.5 3.4 3 •'").... 3.4 3.1 3.6 3.8 _, 1::" ,:') .. ..J 4.3 4.3 18 2.9 -) "7 ...:.. ./ ...., r::-A-+ ..J 2. ~~.i 2.4 2.8 2.4 2.4 2.6 4.1 4.:1. 3.8 3.0 3.4 3.4 4.1 4.0 19 3.3 2.0 2.3 '") ·~ A-. • ..j, 2.0 2.3 2.0 2.0 2.0 4.5 2.3 2.1 '1 .-)...:.. + ..._ 3.3 1.9 2.0 ·'") •'") ...:..+..:.. 20 3.3 3.5 3 .,:.. r) 3.8 3.5 3.1 3.7 2.3 2.4 1.9 2.4 3.9 ~2. :J. 3.:1. 1.9 1.B 2.0 C'Q TABLE 16 ()j MEAN RATINGS OF IMPORTANCE OF 20 ATTRIBUTES N FOR PERFORMING JOB DIMENSIONS AT A SUPERIOR LEVEL D I t1ENG I ON AHIL A B c D E F G H I .J t\ L t1 1 ~, """ 4.7 4.6 4.2 4.1 4.7 4.8 4.6 4 + ,_,c:· 4.3 4.1 ~L6 4.0 3.0 3.2 4.7 4.6 3.4 4.6 3.4 4.3 2.8 3 '").... ~~ + 8 3.2 2.3 3.2 3 4 4.3 4 ~,.... 3.8 3.9 4.3 4.3 4.4 4. :7} 4.0 3.9 4.0 4.0 3.7 3. l 4.3 4.4 4.5 4.4 4.5 4.6 3. :1.3 ,.,....~ 3.0 3.2 3.9 3.6 c· ~~ 6 4.2 3.8 3.8 3.3 4.2 3.5 4.:1. 3.4 3.8 3.2 3.7 3 .l. 4 + !'"j 4.l 3.8 3.4 4.6 3.8 4.0 4.4 3. ~) 4.7 4 + ~) 4.7 3.7 4.B 7 2.8 2.5 3.0 2.7 2.6 2t5 3.5 2.9 4.7 :3. 1. 3.0 :~ + 8 2.7 8 4.4 4.1 4.1 4.5 3.8 3·. 7 4.6 3.9 4.6 4.5 3.6 4.4 3.9 9 4.1 3.8 3.9 4.6 4.0 3. (J 3.7 3.9 4.8 :~. ~5 3.:1. 4. :~ 2.9 10 4.8 3.5 4.5 4. :~ 3.9 3.6 3.6 3.8 3. ~~ 3.7 3.0 2.8 2.4 11. 4.7 3.6 4.3 4.3 4. :1. 3.8 4.3 4 + :1. 4.4 4.6 3.l 3. ::.~ 4 o::·+ ... .1 12 13 14 15 4.8 4.3 4.4 4.4 4.5 4.2 4.2 4.2 4.9 4.2 4.1 4.3 4.6 4.2 4. ~~ 4.4 4.4 4.2 4.1. 4.0 4.1. 3.7 4 ~,.... 4.2 4.0 3.9 4.6 4.6 4.8 4.3 4.0 4.2 4.2 4.0 4.7 4.6 4.7 4.2 4.8 4.7 3. ~j 3.7 4.24 ,.,....._ 3.6 3.5 4.5 4.5 4.1 3.7 4.7 4.5 16 4.0 3.7 3.8 3.8 3.7 3.5 3.6 3.8 3.8 4.2 4.6 4.5 4.1 17 18 3.8 3.2 4.8 3.2 3.4 2.9 3.6 2.9 3.4 2.8 3.6 3 .-,.... 3.6 2.8 3.3 2.8 3.4 3.1 3.9 4.6 4. ~5 4.8 4.5 4.4 4.0 3. :=.:; 19 4.1 '") 1:;" ~·~ 2.7 2.8 2.4 2.7 2.4 2.3 2.3 4.9 2.8 2.5 2.6 20 4.0 4.4 4.0 4.6 4 + ~5 3.9 4. :=.:; 3.2 3.0 '") '") ~. -~ 3.0 4.7 2.4 N 3.8 3.9 3.8 3.8 4.:1. 4.3 4.1. 4 ,., • ...:"'! 4.0 3.2 ~5. 6 4.2 ·4 + 2 4.4 4.9 4.4 4.0 3.9 4.:L 3.9 () ~~. 3 3.4 ~~. :1. 3.4 3.8 3.3 2.9 3.3 4 + :1. '1 t:: ...._ + ~ 2.9 3.8 3.9 4.4 4.2 3.7 3.9 3 + <_;> 2.3 :~. 3 p 2.6 2.9 2. :J· X.lj. ) I j x .. ( y.j I j (I i x .. lj h . h f t he .1th attr1. b ute f or t e j. th d. . were, w..lj = we1g to h 1mens1on X•.lj mean rating of the importance of the ith attribute forperformance on the jth dimension, and y. j = mean rating of the importance of the j th dimension. These weights are shown in Tables 19 and 20 for predicting adequate andsuperior performance respectively. Since these two weighting tablesrepresent the major product of our study, they deserve a good deal ofexplanation. First, note that al 1 values in the main 20 x 16 matrixsum to 100. Thus, each value represents the percentage of total examination points apportioned to predicting performance on one dimension fromscores 6n one attribute. Next, note that the 16 column margi~al values,in the row titled 11 TOT 11 , are exactly proportional to the.mean task dimension importance ratings computed in the previous research phase. Thatis, the highest column marginal value in this row (7. 16 for H) is largerthan the next highest value (7. 10 for F) by the same proportion that themean dimension importance rating for Dimension H exceeded that forDimension F. By apportioning these marginal dimension weights acrossall 20 attributes, we ensured that each dimension 1s weighting in theentire plan.was proportional to its relative importance for the firefighter1s job. Since different numbers of attributes were rated asimportant for each dimension, the percentage of total points assignedto each dimension was distributed across these different numbers ofattributes. For example, in Table 19, since 11 attributes contribute TABLE 19 WEIGHTING PLAN FOR 20 ATTRIBUTES AND 16 PERFORMANCE DIMENSIONS, BASED ON RATINGS OF ATTRIBUTE IMPaRTANCE FOR ADEQUATE PERFORMANCE DI MEN~:; I ON J I< L N () 1..··, 0 TOTAl... ABIL A B c [I E F CJ H I () () () () 0 0 7.3 1 .66 1.94 .91 .~5:1. 2.0~'5 0 0 1..23 0 0 r) .63 0 .96 .50 0 () 0 :1..1.9 • ~:'i3 • 4~'.) 0 0 0 () 0 0 4.3 .:.. () 0 () 0 () 4.03 .5f:l 0 • 83 • 4B 0 0 0 1.1.2 • ~:; 1 .4"7 0 -z r.:· () 0 0 1.17 • ::'i :1. • 4\j> () () () 0 0 0 \.J + ..J 4 () 0 .84 .~50 ,J 0 0 () 0 .~9 1::'" () () 0 • ~'.'j4 () 0 .h.t 0 () () 0 2.1 0 0 () () () 0 0 0 0 .46 1.:1.4 .b6 .G2 0 0 .9:1. 4.0 6 r.:· y ... ) "7 () 0 0 () 0 0 0 0 + ~:; 3 () 0 0 0 0 () 0 8 .60 0 o •4s:· 0 0 1.01 () • ~:=i 1. • 4El () • ..""QJ, .BO () 0 () 4. :=.:;.•::··::·.,_) ..... () 0 () () 0 0 () 1.1 9 0 0 () • :~i2 () 0 0 () () () () 0 () 1.6 10 .69 () .89 () 0 0 0 0 0 0 0 q.::· 0 () 11 • 6~j 0 c,c:· .47 () () t , .. ..1 () .5:1. .4? 0 0 n () 3.9 • ~)J 1 ':> .67 2.05 .99 • ~) 1 2.14 3.56 () l + 2 '7' • 4'? .50 () 0 .83 0 () 1.03 14.1 13 .59 () () .4B () () () 1..1? () () 0 () • i3:l 0 0 0 3. 1 r::····) 14 .61 0 () .48 () 0 :1.. 0 ::~ 0 .54 • ••• ) 1..00 .6.1 .f:l() 5.66 0 .91. 12.2 + 1::"•"")J,~,·_ .91 9.:1. () .~)() J."-0 .63 .94 0 0 () () () () () 0 () 0 0 .4~) J.O~;> ,,6() • 8:5 0 1.90 1.12 6.0 15 .60 0 0 .47 () 3.54 .96 l6 1"7 .58 2.39 () 0 () 0 () () \,') 0 1.:1.0 .b6 .BO 0 1.95 1.:1.2 8 .,:, s::· r::· c:-·"") .,_) • ,_1 18 ¥-,_1 A () 0 0 0 () () () 0 0 :t.J.? • ,.S3 0 0 2.:1.6 .t. o:::; 19 () 0 0 {) () () () 0 () + '"""7..J" () 0 () 0 () 0 .b 20 0 () 0 .50 2.07 0 .97 0 0 0 0 .64 () () () 0 4.2 1:.-.· ..., :::; • <;>() :'5.50 ~J. t'> ,J 6. 6:'5 :'.'i • .S/; ,S.Ol 7.0~5 :I.QO.O TOT 6.i3t> 6.38 6.27 5.9:1. 6.26 7.10 5.90 7.17 ~5. "7 4 ·.~ l..loJ TABLE 20 \.0 ~ WEIGHTING PLAN FOR 20 ATTRIBUTES AND 16 PERFORMANCE DIMENSIONS BASED ON RATINGS OF ATTRIBUTE IMPORTANCE FOR SUPERIOR PERFORMANCE • It I rlEN~:; I ON ABIL.. A B c [I E F G H I .J r:: L N 0 1 2 3 .42 .40 • ~-38 .43 .43 .40 .43 .44 .40 .40 .38 .T·~\.., _., .44 .43 l)•l+ y ...... • 4::'i .49 4 c>• 7 () () -:r LtwO o::··-;+ ,J .I o::·'+ .... J\.) 1::··.,• ...J~) () .43 .42 0 .36 .3? () () 0 0 0 0 .32 .33 .33 0 () 0 4 .3/ .40 .40 .39 .40 .49 0 • :~i4 .41 .38 0 0 .32 0 &::..J 6 .37 .33 • 3'? () .39 .32 + 3!5 0 .39 0 • 4~) 0 .44 .39 .46 0 .43 .35 .34 .37 0 .69 .llo::· • y ,J .47 .35 .37 .60 0 7 8 9 10 :1.1 12 1.., • .:> 14 1r"· . ..J 16 () • 31:/ .37 .42 .4:1. .42 .38 .39 .39 .36 0 .43 .40 0 .3 .7 / .4 -7 / .44 .43 .43 .39 0 •::H1 .36 .41 .40 .45 .38 .38 .40 .35 0 .38 -,. ('"'.._) ., .36 • 3·.? .39 • 3 ,.<) .37 .37 .33 () .39 .42 .4:1. .42 .46 .44 .42 .4:1. + :~8 0 • 4:5 .47 .44 .46 .51 .46 ·~ ..")• ... J "._ + ~:j 2 () .~H .45 .35 • 3~3 • 4 :l • 3'? • 3f.~ + 4~~j .44 • 3!5 0 .48 • 4f.i .47 .,,. ('• ,J ,) • ~58 .53 .49 • ~) 1 .46 .44 .43 .44 () .41 .39 • ~~~7 .44 .43 .3::'i 0 .38 () .31 .39 .39 • ~3~:) .40 .39 .35 0 .52 0 () () .51 • 54 .62 .61 .6? .39 .44 .42 0 () .37 • 3~) .45 .46 .45 .35 .36 • 34 () .3:1. • ~~B .36 .36 . -4:~ .38 0 0 • ,'J!:'i 0 () • 6:1. .6:1. .70 . .:., )' .::59 1? 18 19 .34 () . .,. L • ~=>o .50 0 0 0 0 () .31 () () 0 0 () • ·44 () () • 3~) () () 0 0 () () 0 () .32 .• 39 .41 .65 .69 0 .46 • 4:-5 0 .3:5 .~5:3 .36 .61 .~)3 0 20 TOT .36 6.86 .46 6.37 .37 6.26 .39 5. '71. .44 6.2/' .48 7.12 .44 :s. 90 0 7.16 () 5.74 0 :5.90 0 5.::'j() .47 ::'j. 63 .34 6.66 0 5.67 p 0 () 0 () () () () () () () () .B4 .?8 • 8~5 .?6 • gc,> .89 .99 () () 6.QO (] 0 () 0 0 0 .?0 0 .60 0 0 .59 . ·7-7 / / •6:5 .?0 .70 .80 .78 .76 () () } • ()!5 TOTr"1l_ 3. ~:) 4.3 4. ~5 4.:1. ~5. 4 4.0 1. ~.=; 6.1 ~'). 1 3.2 5.0 7.9 7.4 8.0 7. (jl ?.:L 6.0 4 ,., • A... 1.1 3.8 100.0 95. to performing Dimension A, its 6.86 percentage weighting points are distributed across ll attributes in proportion to the attributes 1 con tributions to Rescue performance. Only one attribute contributes to Dimension 0, on the other hand, so that attribute receives all of 0 1 S 5.66 weight. Although individual attributes thus receive widely varying weights across dimensions, the sum of weights across all attributes will be exactly proportional to the dimensions 1 rcJted impc;rtance. The 11 TOTAL11 columns of Tables 19 and 20 show the total weights assigned to each of the 20 attributes. An overall predicted performance score for each candidate would be computed by multiplying stand~rdized scores on measures of each attribute by these TOTAL weights and summing across attributes. Candidates would be selected in rank order from the resulting list for medical examinations, background investigations, or any other screening steps to be followed in a local jurisdiction. Rank ordering applicants on the basis of weiqhts from Table 19 would indicate their relative potential to perform as competent firefi~1hters. The Table 20 weights make more strinqent demands upon applicants, and should be used primarily in making finer discriminations among people receiving approximately the sam·e scores based on the weights from Table 19. Measures of Attributes. Appendix Q 1i~ts several tests or scales appropriate fo~ measuring the 20 attribute constructs. Test users should select one or more of the measures listed for each attribute, standardize applicants 1 scores on each test, average scores across any tests measuring the same attribute, and standardize these composite scores. The weights' in the TOTAL column of Tables 19 and 20 can then be applied to the 20 standard composite scores for each applicant to identify the applicant 1 s total predicted performance score. lmpl ications of Tests for EEO Laws. The latest draft of the Equal Opportunity Coordinating Council (EEOCC) Proposed Uniform Guidelines for Employee Selection Procedures (EEOCC, 1976) has great ~elevance for this study 1 s results. The EEOCC was established to eliminate procedural inconsistencies among the agencies responsible for enforcing federal equal employment laws. The Equal Employment Opportunity Commission (EEOC), one member of the EEOCC, issued a set of Guide] ines on Employee , Selection Procedures (EEOC, 1970), which are currently in force. These 1970 Guidelines, however, are vague about the requirements for content and construct validation, stating only: [E]vid,ence for content or construct validity should be accompanied by sufficient information from job analyses to demonstrate the relevance of the [test] content or the construct . The types of [tested] knowledges, skills, or behaviors contemplated here do not include those which can.be acquired in a brief orientation to the job. [.S 1607.S(a)]. . 96. Thu~, under existing guidelines, there is no clear statement of when content or construct validation procedures are appropriate. The EEOCC Proposed Uniform Guide! ines (EEOCC, 1976), on the other hand, much are more specific: A selection procedure based on inferences about psychological processes cannot be supported by content validity alone ... ·tohtent validity. is also not an appropriate strategy when the selection procedure involves knowledges, skills, or abilities which an employee wi 11 be expected to learn on the job. [§ l2C(l)]. Apparently then, considering the large amount of technical information firefighter recruits are expected to acquire after entry into the fire service, content validation could be a justifiable strategy only for identifying persons likely to succeed in the training itself. It would, however, be extremely difficult to demonstrate cross-jurisdictional relevance for a selection instrument content validated through such a strategy. This is especially true for departments which have no formal fire training academy, and where the nature of on-the-job training is largely determined by the emergencies which arise during the early period of a recruit's employment. Regarding requirements for the clearly more appropriate construct validation strategy, the EEOCC Proposed Uniform Guide! ines state: (1) [A] job analysis should result in a determination of the constructs that underlie successful performance of the important or critical duties of the job. (2) A selection procedure should be selected or developed which measures the construct(s) [so] identified. (3) [Such a] selection procedure may be used operationally if .. there is suffici~nt empirical research evidence showing that the procedure is validly related to performance of critical job duties. Normally, sufficient empirical research evidence would take the form of one or more criterion related validity studies meeting [the technical requirements for empirical research] [S l2d]. We have accordingly attempted to select as measures of our 20 attributes tests and scales which have been found to correlate with relevant aspecLsof firefighter performance in previous empirical validation studies. Appendix Q includes references to these studies for al 1 such validated tests. In addition, panels of testing and fire service experts will meet 97. in a future research stage to help assess the construct validity of each test as a measure of the attributes it was selected to tap, and in our; final project phase we will assess the tests' empirical validity as predictors of performance in the appropriate task clusters. Since ratings on our Phase V initial ability requirements questionnaire indicated that our 20 basic human attributes are not directly developed after hire, they represent qualification requirements which firefighter recruits must possess at the time they are hired. Phase V raters also reliably indicated that these attributes represent important components of firefighter performance. The instruments in Appendix Q which have been empirically validated against firefighter performance as measures of these attribute constructs may thus claim some degree of construct . validity as entry-level firefighter selection instruments. In our final research phase we shall undertake to demonstrate the empirical validity of many of the remaining instruments. Adapting Procedures to Local Practices, Departments could substitute other tests, including both existing department examinations and newly written tests, as measures of the 20 attributes if the new tests can reliably be shown to assess these attributes. Such a demonstration could be made in two ways. First, if research has shown that test scores correlate significantly with reliable ratings or other measures of standing on the attributes, or with measures of performance in job areas to which the attributes contribute strongly, this would constitute evidence of the validity of the test as a measure of the attribute. Secondly, if a group of people demonstrably expert in the areas of fire service or psychological testing reliably indicate that the test is a measure of the attribute, this would indicate at least a moderate degree of construct validity of the test for measuring the attribute. The method employed to make this judgment should include obtaining ratings, or some other quantitative data, amenable to reliability analysis. Lawshe (1975) ,·for example, describes a method of evaluating the degree of consensus in panels of expert judges making simi Jar decisions. It is highly recommended that both approaches be used, with experts identifying the attributes apparently tapped by a test, and empirical research confirming their judgments. 98. XI. USE OF JOB ANALYSIS RESULTS IN OTHER JURISDICTIONS Background. The extremely high rel iabil ities we obtained throughout all phases of our study indicate excel lent consistency among raters from all over the country, and the high inter-region and inter~size correlations (typically above .80) show that no systematic departmental differences exist across these factors. These results, taken together with the size and distribution of our sample, suggest that selection procedures arising from our job analysis ~1ould be applicable to any paid department in the United States with at least 25 to 50 personnel. lt. is necessary, however, to demonstrate similarity between firefighter jobs in departments seeking to use the results of our study, and those in the jurisdictions which participated. Regarding this subject, the EEOCC (1976) Proposed Uniform Guide! ines state: It is the intent of the agencies issuing these guide! ines toencourage and facilitate cooperative development and validationefforts by employers, labor organizations, and employment agenciesto achieve selection·procedures which are consistent with theseguide! ines. [g6a] Selection procedures shown by one user to be content valid ...will be considered acceptable for use by another user for a performance domain if the borrowing user's job analysis shows thatthe same performance domain is measured by the selection procedure.[s 6c] A full descrip(ioQ should be provided of the similarity betweenthe performance domain in the.user's job and the performancedomain measured by a selection procedure developed and shown tobe content valid by another user. The basis for determiningthis similarity should be explicitly described, [i13e(l2)i] Where a selection procedure satisfies the.standards [for constructvalidity], it may be used operationally for other jobs which are shown by an appropriate job analysis to include the same construct(s)as an essential element in job performance. [&12d(4)] A full description should be provided of the basis for determiningthat the construct identified as underlying successful job performance by. the user's job analysis is the same as the constructmeasured by the selection procedure. [S l3e(3)i] Thus, to justify using our results, other jurisdictions must demonstrate.that the tasks performed by their firefighters are essentially the sameas those performed by firefighters in our developmental group. Webelieve that the high level of reliability achieved within a nationally 99. representative sample of job experts for the ratings linking 20 human attributes to performance on 16 important task clusters, demonstrates the relevance of those attributes for any job comprising the same set of tasks. Thus, a new department not previously involved in our project can justify using the selection measures identified in the previous section, by demonstrating that its firefighters perform substantially the same tasks as those performed by the departments in our sample. Specifically, we recommend that new departments complete a task check 1ist similar to the one we used in the task analysis constituting Phase I I of our study. By comparing the resulting profiles of mean task importance values with the profiles defined by our natiQnal Phase I I sample, are new departments can ascertain whether their firefighter positions similar to those of the departments used to develop our recommended procedures. To reduce the number of comparisons to be made to a reasonable number, and to relate the job similarity assessment to the performance scales to be used in our subsequent validation effort, we recommend col lapsing ratings for 109 important task items into the 16 performance dimensions identified earlier, and assessing job similarity between new and development sample departments across this 16-point profile. One advantage of this procedure is that it permits departments to ignore test scores validly predicting performance in those job areas which their firefighters do not perform, but to use the balance of the selection battery in their screening procedures. Technique. To assess the similarityDeveloping Job Similarity Assessment of new departments 1 mean dimension importance profiles to that of the development sample, we will test whether a new department 1 s mean rating for any dimension deviates from the development sample 1 s means more than would be expected by chance at the five percent level. By assuming that the 48 departments in our Phase II sample const'itute a representative sample of U.S. fire departments, we can conclude that the five percent most extreme Phase I I department cluster rating means represent the deviations from the mean prefile expected by chance at the five percent level. Using as cutoffs the mean cluster ratings of the departments falling at the 2.5th and 97-5th percentiles for each cluster then produces a probability of .05 that a new department from the same population wi 11 exceed these cutoff points for each cluster. Since 2.5 percent of 48 departments is 1.2 departments, we considered the two most extreme scoring departments in each direction from the total sample mean. In general, the mean importance-time spent cross-product rating for the most extreme department defines the .025 cutoff value for each task cluster. 100. Where a large interval exists between the most extreme and the next most extreme departments, however, we recommend placing the simi Jarity confi dence I imit at the mean rating value of the latter, so that extremely deviant responses will not set an overly wide acceptance band. Figure 12 shows the mean importance rat-ing profiles based on the most extreme and the next most extreme departments in our development sample for each cluster. The heavy I ines at the top and bottom of Figure 12 represent the maximum and minimum possible mean rating values to give an indication of the relative width of the acceptance range. A seriously large dis crepency between the most and next most extreme departments occurs only for the lower limit on Dimension F. Thus, we recommend defining an upperconfidence limit for assessing new department similarity to our development sample by the highest department mean rating for each cluster. The lower similarity confidence 1imit is defined by the lowest department 1 s mean rating value for each dimension except F, and by the second lowest department 1 s mean for F. These limits are shown in Figure 13. This procedure constitutes a synthetic validity approach to selection(see Guion, 1965), in which new departments may use the tests for allclusters whose importance cutoffs their mean ratings do not exceed.If a new department 1 s ratings indicate they perform the same taskswithin a cluster as did our development sample, then the attributesrequired by the new department 1 s firefighters for that cluster have beenidentified, and the department should use the tests selected to tap thoseattributes. Since most attributes cut across several dimensions, thedepartment should weight standardized scores on the appropriate testsby the values shown in the proper columns of Table 19 or Table 20.Weights from columns representing those dimensions whose importancecutoffs were exceeded shou 1 d riot be used. If the departments desire,the weights actually to be used could be rescaled to sum to 100 bysumming alI such weights, dividing 100 by the sum, and multiplying eachof the weights to be used by the quotient. This in no way affects the·relative emphasis given to each attribute and each dimension, but itdoes permit a simple evaluation of the percentage of total scores represented by each attribute. Procedure for Assessing Department Simi•Jarity.' To determine for whichclusters it is sufficiently similar to our development sample to justifyusing our results, a new department must perform an abbreviated analysisof the tasks its firefighters perform. It should have a checklist containing the 109 tasks which define the 16 clusters completed by at leastten departm~nt members, including firefighters with at least two yearsexperience, company-or chief-level officers who directly supervise fire fighters, and additional department personnel who ar~ exceptionallyfamiliar with the work performed by firefighters. Ten raters were estimated,on the basis of our Phase I I data, to be the number required to produce aHoyt rei lability between .70 and .80 across the task ratings. Each task 24 22· 20 18 16 14 12 10 8 6 4 2 22 20 16 14.. ...•' l2 10 8 6 24 22 20 22 24 20. 22 22 . 22 24 22 22 20 18 20· 20 20 2022 20 22· 20 16 20· 18 18· 18 18· 16-18 1820 18 20 24 18 18 16 14""16 18 16 16· I 16 14 16 I 16 18 22 16 16 14 . 14 14 I ,'~ 12 14 ,• .. 16 ' ...' 1 .... -----/I "·J 4 ,. ···~o / ' .... , ... •• 14 ... 1 12 10 12 14-\ ...... ' .....f../ "1.2' / ,/14 . ~ ••.. .."12 ./ ~ .... .... 14 18· 'I -8 12 10 10 10 10 ;1 10· 10 i 12 12 16 lQ. 8 6 10 10 8 6· 8 8- 8i 8 10 .J 10 14 8· 4 8 8 6 6 4· 6 8 6 12 61 6 6 6 8 4 ~ I,/ /,' I 2 4 Q H ---------= most extreme departmentsc -----= second most extreme departments F FlgQre 12. Plot of task cluster importance-time spent cross-product means for most extreme and second most extreme departments for 16 clusters. 24 22· 24 22 20 24 20· 22 20 22 22 24. 22 22 20 20 22 18 20 18 20 22 20- 20 22' 20- 20 I8 16 20 I6· 18· 18 I8 16 20 I8 18 20 24 18 I8 I6 I4.. 'I 8 I4 16 I6 16 16 16 ( ( .' I 4 16 18 22 I ' ' 16 ( ' 16 ,'14 'l2 T6- I4 12 .... .......14 ..... •"'I 6... ... ...•, 14 .. -·· --·-zo ...... _14 ' 14 . 14 , .' --------.---'•'•I 4 I2 14 10 12 I2 ---14 12 ' ...'.' .,/12,. I 2 10 I2 14-••' _...ro-...... "l~ "' 14 18· 12 12 10 8 I2 8 10 IJ lo. 8 10 I2 ! 10 10 I 2 I6 IQ 10 8 6 10 6 8- 8 aj 8 101 8- 6 8 10 I 4 j 8- 8 6 4 8 4 6 6 4 6j 6 6 6 8 I2 .... .. 8 6 4 2-6 6 2 .. 4 .. 4 4 1t •.... 6 I ,' .-.' ....6 lq ----........... -..........4. ,,' ' '.. ~ 4 .....-·2-.' 4 I .. ' , 2 --.. 4 I ' I ''• 4 ... . .. ' 4..... ' I 4.."' 2 I ~-' .·----.... 2. 8 2 Q H N --------= recommende-d cutoff profi I e c F Figure 13. Recommended cross-product task cluster importance cutoffscores for identifying departments similar to development sample. 103. should be rated for the amount of time firefighters spend performing it, and for. its relative importance to achieving fire service objectives. For each rater, the department should compute the product of time spent and importance ratings for each task, and average these values across the tasks included in each task cluster. These importance means should, in turn, be averaged across the ten raters to obtain a total department mean rating for each cluster. The mean rating for each of the 16 clusters would then be plotted on a g~aph like the one shown in Figure 13. If the plotted mean for a cluster 1ies between the two confidence limit points for that cluster, the department may be considered sufficiently similar to the departments in our development sample to warrant ~sing the selection instruments we identified for the cluster. Weights from Tables 19 or 20 should be used to weight and co~bine scores on the instruments across all clusters relevant to the department. Appendix R conta1ns the materials and instructions required for this operation. 104. XII. SUMMARY AND CONCLUSIONS The literature we reviewed in Section I I showed several trends in current firefighter validation research. Most studies were performed in individual jurisdictions or within restricted geographical areas. Typically, because specific job analysis methods and norms were not described, it is difficult to extend research findings to other depart ments. Frequently no empirical test was made of the validity of recom mended selection procedures, although the most recent draft of the EEOCC Proposed Uniform Guidelines suggests that a pure content validation is inappropriate for those job aspects in which firefighters receive exten sive training. Where empirical studies were performed, they frequently employed multivariate regression techniques with limited sample sizes and no cross validation. To avoid at least some of these problems, POl contracted with the U.S. Civi 1 Service Commission to perform a nationwide validation study of the entry-level firefighter position. We included a total of 109 juris dictions from all areas of the country and from a wide range of department sizes. We carefully planned, performed, and documented our researchactivities to ensure that our data would be maximally useful and ourresults maximally replicable. We performed a one-year job analysis studyinvolving all of our 109 jurisdictions, to ensure that all important firefighter tasks were considered, to identify subgroups of tasks based onindividual differences in performance, and to identify the contributions of various human traits to performance on. these task subgroups. Thesetraits or attributes represent relatively stable human abi 1ities or aptitudes, rather than specific skills readily amenable to instruction.In future project stages, we will empiri~ally test the relationshipbetween these constructs and actual measures of performance in a nation ally representative sample of departments. \>/e have developed norm·s, methods, and specific instructions for determining whether departmentsnot participating in our study are sufficiently simi Jar to our develop ment group to warrant using the results of our project. It has been our intention to conduct research that will be broadly anddemonstrably applicable to~· large number of fire departments in theUnited States. Because of the invaluable assistance, encouragement, cooperation, and conscientious effort of more than 500 members of 100 departments, we believe this study has progressed considerably towardthat goa 1. 105. Reference Notes 1. Cecil, C.E., and Cecil, J.H. Interim report on recruitment and validation of written examinations useJ in hiring and promoting employees of the Montgomery Cit -County Personnel Board -fire department. Undated, unpublished manuscript. Available from C.E. Ceci I, Department of Ecucational Psychology, University of Alabama, Tuscaloosa, AL 35486.) 2 .. Dallas Regional Commission. Job analysis study, entry-level fire fighters. Dallas: Dallas Regional Commission, U.S. Civil Service Commission. 3. Rusmore, J. The relevance of mechanical ability to firefighter performance. Unpublished manuscript, San Jose State University, 1974. 106. P.EFEHENCF:S Allen, D., Bodner, W. S., Lano, R. L., & Meyer, J. M. A study of thefireman 1 s occupation. Bureau of Industrial Education, CaliforniaState Department of Education, and Division of Vocational ~ducation; University of California, 1973. Alvares, K. M., & Boston, D. A. The validation of a selection device for metropolitan firefiohters. (Technical Report ~!o. TCSC7h"-l) Toledo: Toledo Civil Service Commission, 1976. Atlanta Regional Commission. A study of applicant selection and performance appraisal for firefighters, pol ice officers, clerical employees,and child c'are attendants (Two volumes). Atlanta: Atlanta ~eqionalCommission, U.S. Civil Service Commission, 197lf. Bartlett, M. S. Multivariate analysis. Journal of the Royal StatisticalSociety, Series B, 1947, 2_, 176-1~7. Bloom, B. $. (Ed.). Taxonomy of educational objectives, cognitive domain.New York: David McKay, 1965. Borman, H. C. The ratinq of individuals in organizations: An alternate approach. Orqanizational Behavior and Human Performance, 19711, Jl_, 105-124. Brumback,(). B. Romashko, T., Hahn, C. P., &·Fleishman, E. A. ~1odel procedures tor job analysis, test development, and validation \AIR. report AIR-376-00-lf/?lt-FP.). \•'ashinqton, D.C.: American Insti tutes fo~ Research, 197L. Campbell, J. T. Principal results of the study and conclusions. In L; A.Crooks (Ed.). An investiaation of sources of bias in the predictionof job performa~ce. Princeton: Educational Testing Service, 1972. Christal, R. E. The United States J.lir r-orce Occupational Research Project(Technical Report No. AFHRL-TR-73-75). Air Force Human ResourcesLaboratory, Occupational Research Division, Lacklan~ AFB, Texas, 1974. Dunnette, M. D. Personnel select.ion and placement. Belmont, Cal if.:Wadsworth Publishinp, 196~. Equal Employment Opportunity Commission. Guidelines on emnloyee selectionprocedures. Federal R.eaister, 1970, 35_ (14.9), 12332-12336. Equal Employment npportunity Coordinating Council. Uniform guidelines onemeployee selection procedures. Federal Reqister, 1976, ~-(136)29016-29022. 107. Fleishman, E. A. The structure and measurement of physical fitness. Englewood Cliffs, N.J.: Prentice-Hall, 1964. Guion, R. M. Synthetic validity in a small company: A demonstration. Personne 1 Psycho 1ogy, 1965, ~. 49-63. Heckman, R. \-1. Grand Rapids firefighter test validation study. Minneapolis: • Personnel Decisions, Inc., 1973 a Heckman, R. W. Saint Paul firefighter validation'study. Minneapolis: Personnel Decisions, Inc., 1973Tb). • Holland, J. L. Manual for the vocational preference inventory. Palo Alto, Calif.: Consultinq Psychologists Press, 1965. Hoyt, C. J. Test reliabl ity obtained by analysis of variance. Psychometrika, 1941' ~. 153-160. International City Management Association. 1975 Municipal Yearbook. lCMA, ]140 Connecticut Avenue N.W., Washington, D.C. 20036. 1975. Johansson, C. B., & Campbell, D.P. Stability of the Strong Vocational Interest Blank for Men. Journal of Applied Psychology, 1971, 55, 34-37. Lawshe, C. H. A quantitative approach to content validity. Personnel Psychology, 1975, ~. 563-575. Marks, L. G.: An exploratory study of behavioral characteristics of certain selected municipal firefighters ut-ilizing the Personal History Index (Doctoral ~issertation, University of Southern California, 1970). Dissertation Abstracts lnternati,onal, 1970, ~. 3625A ~University Microfil~s No. 70-26, 527). Marquardt, D. C.,[. McCormick, E. J. Attribute ratinqs and profiles of job elements of .the Position Analysis O_uestionnaire (PAQ). Lafayette, Ind.: Occupational Research Center, Department of Psychological Sciences, Purdue University, Report ~lo. 1, 1972. McCormick, E. J.", Jeanneret, P. R., E. Mecham, R. C. A study of _job c-haracteristics ar1d job dimensions as based on the Posit ion Analysis Questionnaire. Lafayette, Ind. Occupational Research Center, Purdue University, Report-No. 6 :, 1969. Murdy, L. B., & No.rton, R. P. Fire private test validation study (IBR Tech. Report No. 72-'2). Fort Worth: Texas Christian University, Institute of Behav iora 1 :Research, 1972. National Fire Protection Association. Firefighter professional qualifica-: tions (NFPA Nc1. 1001). NFPA, 470 Atlantic Ave., Boston t1A 02210, 1974. 108. Personnel Decisions, Inc. Report on test valida·tion study, Minneapolis Civil Service, firefighter jobs. Minneapolis: PDI, 1971. Pr i mo f f , E . S . How to prepare and conduct Job Element examinations (Technical Study 75-1). Research Section, Personnel Research and Development Center, U. S. Civil Service Commiss1on, Washington DC, June, 1975. Romashko, T., Brumback, G.B., Fleishman, E. A., & Hahn, C. P. The development of a procedure to validate physical tests. Washrngton, DC: American Institutes for Research, 1974. Ruedebusch, V. M., & Keck, M. Physical strength and agility examination: Development report and administration manual. St. Louis: St. Louis Regional Commission, U. S. Civil Service Commission, 1975. Ruedebusch, V. M., Vaaler, T. P., K2ck, M., Powers, M., Mendenhall, M., Bayreder, C., Black, P. A study of the validity of proposed! written examinations for the selection of entry-level firefighters in Des Moines. St. Louis: St. Louis Regional Commission, U.S. Civil Service Commission, 1975. Selection Consulting Center. The validation of entry-level firefighter examinations in the states of California and Nevada. Sacramento: Selection Center, 1973. Selection Consulting Center. The selection of entry-level firefighters, Phase II: Interviews and physical performance. SacramenJto: Selec tion Consulting Center, 1975. Smith, P. C.,.& Kendall, C. M. Retranslation of expectations:; An approach to the construction of unambiguous anchors for rating sc~les. Journal of Applied Psycholoqy, 196~, ~. 149-155. Tice, T. E. Selection systems and performance appraisal in the fire service: A study of criterion development and test validation. Unpublished masters thesis, Iowa State University, Ames., Iowa, 1970. Tornow, W. W., & Mabey, D.S. The development of a classification system for management jobs at CDC (Personnel Research Report· No. 59-75). Minneapolis: Control. Data Corporation, 1975. U.S. Civil Service Commission. Achieving job-related selection for entrylevel police officers and firefighters. U.S. Civil Service Commission, Bureau of Intergovernmental Personnel Program!; and Personnel Research and Development Center, Washington, D.C., 1973. Verducci, F., & Meekins, T. Proposed physical aptitude examination forSan Francisco firemen. Unpublished manuscript, Sar1 Francisco State University, 1973. 109. Ward, J. H., & Hook, M. E. Application of an hierarchical arouping procedure to a problem of grouping profiles. Educational and Psychological Measurement, 1963, ~. 69-81. \;ernimont, P. F., & Campbell, J .. P. Siqns, samples, and criteria. Journal of Applied Psychology, 1968, ~. 372-376. West Valley Community College. Model assessment laboratory: Firefighter selection examination developed for the City of Palo Alto, Cal lfornia. Campbell, Calif.: Center for Supervision and Management Development, West Valley Community College.· APPENDIX A DEPARTMENTS PARTICIPATING IN NATIONWIDE ENTRY LEVEL FIREFIGHTER JOB ANALYSIS Study Study Department Phases''' Department Phases,., New Haven CT l ,3 ,5 Syracuse NY 2,3 '5 Lowell MA l , 3 '5 Auburn NY 2,3 ,5 Charlotte NC l , 3 '5 Alexandria LA 2,3,5 Columbia l .3 '5 Colorado Springs co 2,3,5 Decatur GA 1,3 '5 Pueblo CO 2,3,5 Toledo OH l ,3 '5 Littleton CO 2,3,5 Oklahoma City OK l ,3 ,5 Santa Clara CA 2,3 ,5 Sutter County CA 2,3 ,5 Lubbock TX l '3 '5Stillwater OK l '3 ,5 Salem OR 2,3,5 San Die~o CA l ,3 '5 Bellingham WA 2,3 ,5 Riverside CA l ,3 ,5 Renton WA 2,3,5 Chula Vista CA 1,3 ,5 Utica NY 2,3 Baton Rouge LA 2,4 ,5 Spokane WA l '3 '5Boise r:ity ID 1,3 '5 Omaha NB 2,3 Pullman WA l ,3 '5 Sioux City lA 2,3,4,5 2 Davenport lA l Fremont NB Marion IN I , 3 Santa Ana CA 2,3 Pontiac Ml 3 '5 Providence Rl 2,4 Natick MA 3,5 Harrisburg PA 2,L Mel rose MA I Montgomery County MD 2,4 Pittsburgh PA 2 Savannah GA 2,4 Can ton OH 2 Athens GA 2,4 Johnstown PA 2 San Antonio TX 2,4 Miami FL 2 Beaumont TX 2,4 Hialeah FL 2 Orange TX 2,4 Coral Gables FL 2 Oakland CA 2,4 Wichita KS 2 Salinas CA 2,4 Topeka KS 2 Fairfield CA 2,4 Salina KS 2 Seattle WA 2,4 Odessa TX 2 Everett WA 2,4 Carlsbad NM 2 Bremerton WA 2,4 San Jose CA 2 Cedar Rapids lA 4 Bakersfield CA 2 Marsha I l town lA 4 Santa Cruz CA 2 Greenville SC 2 Portland OR 2 Corvallis OR 2 *See Page A3 for description of phases. A2. Department Norfolk VA Chesapeake VA Danville VA Philadelphia PA Williamsport PA Ci n c i n nat i OH Middletown OH · Sprinqfield OH Salt Lake City UT Ogden UT Los Angeles CA Fu 11 erton CA Lynwood CA Tucson AZ Scottsdale AZ Prescott AZ Study Phases,·~ 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Department Jersey City. NJ Clifton NJ West. Orange NJ Louisville KY Milwaukee WI Wauwat osa ~/ I Menomonee Falls WI Albuauerque NM Long Beach CA Inq Iewood CA Billings MT Great Fa II s MT Bozeman .MT Boston MA Lawrence MA Nashville TN St. Louis MO Temple TX San Francisco CA Richmond CA Millbrae CA Eugene. OR Springfield OR Study Phases,·~ 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 *See Page A3 for description of phases. A3. Phase Activity . 1 Job analysis site visits to 18 departments. 2 Task analysis checklist mailed to 51 cities, returns obtained from 230 members of 48 cities. 3 Task similarity rating sessions scheduled for 108 members from 54 departments, attended by 93 representatives of 47 departments. 4 Two-day workshops to obtain information and pretest survey materials, 31 representatives from 16 departments attended. 5. Job information surveys mailed to 68 cities with 53 returns. APPENDIX B AGENDA AND CHECKLIST USED DURING PHASE I SITE VISITS TO 18 FIRE DEPARTMENTS SITE VISITS FIREFIGHTER STUDY 1. Explain study to those who participate a. USCSC is funding us to do this 18-month study which began July 1 b. Objectives c. Sample -126 citie~ -phases -how they were chosen (randomly) d. Discuss the other phases involving them e. Mention payment for workshops in which they will participate f. Appreciate their cooperation g. Benefit~ to them 2. a. Review the day 1 s schedule b. Answer questions 3. ln.terview with Chief or high-level official a. Number of uniformed firefighters by job title b. Number of stations, shift times c. Levels within job ciassification of firefighter d. Similarity of duties, specialities (emergency, fire inspection, equ1pment operation, communications, etc.) e. Major duties/tasks of firefighters in task oriented, behavioral terms f. Importance, time spent, frequency, complexity of duties 11 f 11 above, time permitting) g. Subtasks under duties (then repeat h. What conditions and circumstances influence performance of the duties (like making a duty more difficult on some occasions and less difficult on others) B2. 1. Volunte'er vs. full time j. KSOAs required 4. Interview with training director a. Discuss length and content of training program b. Review and/or get copies of training materials if possible c. Review proficiency tests used .in training d. One-half hour tour of training facility/classrooms e. Discuss major tasks/duties f. Di.scuss what qualifications applicants bring (should bring) to the job and t1ow people change on KSOA during classroom and onthe-job training 11 311 g. Repea.t relevant questions in above, time permitting--don 1 t be redundant 5. Interview with Captain or high-level station commander at a fire station 6. Have one incumbent firefighter (with 2-8 years of experience) and one first-level supervisor complete task check list (the supervisor should complete the check list for the firefighter job, not for his own) a. Expl.ain purpose of our study b. Indicate how the check 1ist was devised c. Indicate purpose for giving them the check list a. Explain study b. Repeat 11c-h11 and 1 T 1 from 11 311 c. Tour the station 1. to see if the list is comprehensive B3. 2. to see if tasks can be eliminated 3. to get a rough idea of time spent and importance of each task d. Emphasize that this is confidential and has nothing to do with their department, job evaluation, reorganization, etc.--their responses will be combined with responses of 17 other departments to help us prepare a mail-out questionnaire e. Go over instructions with them (front page of check list) 7. Be alert to future phases ot the project during these visits. Generate ideas about differences in duties/tasks by department size and geographic region. If possible, find out how many minorities work in each department and whether they are assigned different duties. Think about possible predictors and criteria. Find out how the department currently selects firefighters and what rol~, if any, Ci~il Ser~i~e plays. 8. Exit interview with the Chief or 1iaison official B4. FIREFIGHTER TASK CHECK LIST Our firm, Personnel Decisions, Inc., has received a contract from theUnited States Civil Service Commission to study the job of entry levelfirefighter across the country..To give ourselves a place to start from,we have reviewed all the studies of firefighter jobs that we could find,and we have developed~ list of all the tasks or duties we found mentioned in any of these studies. We have tentatively broken these downinto nine different sections, each dealing with a different part of thejob. We would I ike you to work through this 1ist of tasks and do twoihings. First, so we can get a rough idea of how important each taskis, please indicate with a 11 +11 or a 11 o11 whether an entry level firefighter in your department does or does not perform each task on a reasonably regular basis. Second, at the end of each section, please stop anduse the blank pages at the end of the 1ist to write down any additional tasks that firefighters in your department perform, but that were not included here. Please work very carefully at this, so we can be sure wehave covered the job completely. By collecting work sheets I ike these from your department, and from. 17other departments across the country, we hope to get a pretty good pictureof what the typical firefighter has to do on the job. Then we can developstandards of how wei I a new firefighter has to be able to perform eachtask, and what sorts of examinations might be given to be sure thatapplTcants are qualified to perform this crucial job. Thank you for taking the time and effort to complete this task carefully and thoroughly. + if done BS. 0 if Tasks 1. Maintaining Building and Grounds at the Station. a. Housekeeping chores 1. clean floors, walls, dishes, quarters, oven, refrigerator, etc. 2. b~y food and prepare meals , b. Inspect and repair or construct roofing, plumbing, glazing, heating equipment, etc . .c. Maintain outsi~e grounds 1. cut grass, shovel snow, wash sidewalks, display colors d. Safeguard fire department property from theft, vandalism, damage, etc., at the station. 2. Inspect, Test, Maintain, Clean, and Repair Fire Department Equipmentand Apparatus. a. Vehicles and fire truck on all systems and parts b. Hand tools c. Breathing masks and equipment d. Hoses, coup! ings, and nozzles e. Hydrants f. Fire alarm equipment g. First aid equipment h. Salvage covers and tarps i. Ladders and ropes j. Pumps (gas, oil, and water) k. Batteries and portable 1ighting systems 1. Personal firefightJng equipment (e.g., boots, coats, etc.) m. Leather straps, belts, etc. n. Rescue equipment (e.g., stretchers, block and tackle, etc.) o. Fire extinguishers p. Perform repair operations 1. using lathe 2. welding 3. forming and fabricating parts 3. Responding to Fires a. Receive, process, and transmit alarms b. Drive apparatus to and from scenes of fires by the shortest route 1~ choose route, read map 2. ·operate siren and 1ights 3. drive vehicles 4. position apparatus at the scene ~. + if done Tasks 0 if c. Evaluate and monitor fire conditi ~sand select appropriate procedures . 1. observe fire conditions 2. notify chief officer and others of new information on conditions 3. locate fire source 4. select appropriate equipment and procedures 5. decide where to locate equipment, apparatus, ventilation openings, escape routes, etc. 6. decide whether to request additional or release unnecessary equipment d. Gain entry to buildings or vehicles, forcibly or otherwise e. Ventilate buildings or vehicles 1. make openings 2. place exhaust fans f. Lay hose 1ines 1. rol 1, carry, drag hose g. Make and unmake hose connections and coup] ings 1. to water source (e.g., hydrant) 2. to pump 3. to nozzles or appl lances 4. to other hoses h. Operate water sources 1. hydrant 2. water tower 3. tanker i. Operate pumps lo monitor gauges 2•. operate valves 3. solve hydraulics problems using mathematics j. Use ladders 1. maneuver, raise, lower, and hold portable ladders 2. operate aerial ladders and elevated pla~forms 3. climb ladders, often carrying people or equipment 4. work from ladders k. Extinguish fires 1. operate fire extinguishers 2. operate hose 1ines 3. operate foam generators, proportioners, and lines 4. operate sprinkler systems 5. beat grass fires with broom, shovel, etc. 6. use explosives to extinguish fires 7. ·operate ladder pipe on aerial ladder 1. Confine fires to smallest possible area m. Tie knots n. Lift or hoist objects using levers, ropes, or blocks and tackles o. Operate portable generators and/or lighting equipment + if done 87. Tasks 0 if p. Use portable breathing apparatus q. Operate radio eq~ipment en route to and at fire scene r. Locate and operate shut-off valves for gas, electricity, water, or oil lines s. Control traffic at the scene of a fire · t. Safeguard property l. pile and cover valuables 2. cover flnors and walls 3. carry furniture, etc., from buildings 4. prevent thefts u. Remedy equipment failure l. perform makeshift repairs, improvising with other tools or equipment 2. replace burst hoses v. Supervise others w. Use power and hand saws, boring and drilling tools, 1ifting and prying tools, and digging tools x. Overhaul premises after a fire 1. detect and extinguish hidden fires 2. remove debris from building 3. cover or close openings made during fire 4. place premises' in a safe condition 5. remove water from building 6. restore sprinkler system to service 7. provide means to prevent unlawful entries or thefts 8. turn in any personal property found y. Replace equipment on apparatus pri~ to leaving the scene 4. Rescuing People and Giving First Aid a. Locate victims b. Remove victims from entrapments (such as vehicles) c. Remove victims from burning buildings l. man 1ife net 2. carry victims down ladders and/or stairs 3. remove victims using drags; carries; cots; scoops; ch~ir stretchers; evacuators; life guns, lines, and belts 4. lower or hoist with rope after tying knots d. Transport from the scene e. Diagnose injuries and plan treatment f. Treat injuries 1. treat various types of shock 2•· res tore breathing 3. treat heart malfunctions (including cardiac massage) 4. treat burns 5. control bleeding 6. treat poisoning or drug overdoses 7. treat injuries to bones, joints, ahd muscles B8. + if done Tasks 0 if 8. treat convulsions, fits, and seizures 9. treat for exposure to extreme temperatures 10. treat for smoke inhalation 11. treat eye injuries 12. assist in childbirths g. Handle corpses 5. Investigating Fires a. Determine origin a~d cause 1. detect and protect evidence b. Conduct arson investigation 1. collect evidence 2. interpret evidence 3. process evidence through prosecutor or other official c. Estimate fire losses 6. Inspecting Premises a. Inspect residential and commercial buildings for fire hazards and code violations b. Inspect storage and handling of hazardous materials c. Inspect fire prevention and firefighti"ng equipment in buildings 1. standpipes 2. sprinkler systems 3. fire extinguishers 4. detection and signaling devices 5. fire escapes, ladders, stairways d. Inspect fire alarm systems e. Inspect heating, cooling, ventilation, and electrical systems f. Obtain and record information for reports and prefire planning(e.g., escape routes, access to areas within buildings, etc.) g. Investigate complaints of fire dangers h. Develop methods of eliminating or safeguarding hazards and obtain property owners• compliance; (report refusals to comply) i. Inspect locations where citizens desire to burn materials prior to issuance of burning permits j. Drive through residential and business areas looking for code violations ]. Training Self and Other Department Members a. Learn, read, and study information related to firefighting on individual basis b. Plan and conduct practice drills c. Train new recruits and coworkers d. Participate in critically evaluating each fire e. Participate in training courses or sessions B9. + if done Tasks 0 if 8. Performing Clerical and Administrative Tasks a. Maintain files and records b. Make written reports l. fire alarm reports 2. inspection reports (including prefire plans} 3. fire reports 4. daily log 5. investigation reports c. Maintain inventory of supplies d. Prepare correspondence e. Operate office equipment f. Perform personnel functions 1. payroll 2. locate, apply, and explain civil service regulations g. Plan, organize, or coordinate fire administration programs 9. Meeting the Pub! ic a. Conduct fire station tours b. Explain inspection program, safety hazards, and fire regulations c. Explain firefighting duties and services d. Represent the fire department at community assemblies and meetings e. Demonstrate fire equipment f. Inspect bicycles, issue tags, and collect money g. Control spectators h. Comfort distraught persons i. Answer telephone and provide information to callers 1. assist visitors j. Conduct fire drills k. Give Boy Scout firemanship tests 1. Promote good community relations m. Adjust citizen complaints n. Participate in fire prevention details to places of public assembly APPENDIX C PHASE I I FIREFIGHTER TASK ANALYSIS CHECKLIST November 1975 PERSONNEL DECISIONS, INC. C Our thanks and compliments to the Littleton, ~olorado F.D. 251 5 F'O S H A Y T 0 W E R MINNEAPOLIS. MINN. 55402 INSTRUCTIONS FOR THE FIREFIGHTER CHECK LIST Introduction Our company, Personnel Decisions, Inc., is a Minneapolis-based firm of industrial psychology consultants. We have received a contract from the S. Civil Service Commission to study the job of Firefighter in a U. variety of communities across the country, and we are asking you to help us in this study by providing some information about the Firefighter job in your department. By "Firefighter" we mean firefighting personnel who have completed formal For examp 1e, training and probation, but who ·have not yet been promoted. in some departments, .promotional positions have been crea'ted for drivers of apparatus, and these duties are no longer part of the firefighter•s job, except to fi 11 in for an absent driver in an emergency. In other departments, all firefighters are expected to handle the apparatus, and the members of a company rotate regularly through all positions and functions. We would like you to tell us what firefighters in your department are expected to do as part of their job. The work performed by a firefighter includes routine work around the station, caring for apparatus and equipment, fire prevention and inspection duties, training activities, paperwork, and public relations activities, as well as actual firefighting, rescue, and first aid duties. We are going to ask you t,o use the attached check 1ist to indicate the amount of time a firefighter in your department spends on a variety of tasks in each of these areas, and to indicate the importance of each task for successful performance as a firefighter. To make these ratings, you will need to know just what we have included here in our description of a firefighter•s job, so stop now and quickly read through the entire check· 1ist once without makrng-aii'Y""ratings, so you can get a feel for what is included. After you have read through the list, please continue with the instructions. How To Complete This Check List First, please fill in all the information we have requested on page 5. This will enable us to identify differences in the duties of firefighters across various departments and across different regions of the country. Next, work through the check list and make the following ratings for each task: If firefighters in your department never perform the task, put an "X" in the first column, "Never Perform,•• and go on to the next If they do perform the task, but only very occasionally on a task. relief or stand-in basis (for example, only when a driver or an officer C2. is sick o~ on vacation and cannot be replaced), please indicate this with ''X11 an in .the second column, 11 Rel ief Only.&' Next, for these ''Rei ief Only11 tasks, and for alI tasks regularly and normally performed by your firefighters, pI eas.e 1"r i te the number (1-5)_ correspond Lng to the amount of time spent performing each task in the ''Time Spent11 box on the page. Please note that we are interested in the time spent relative to time spent on ~other tasks performed, so try to take all 204 tasks--listed here into account in rating each task. Time Spent ImportancE' lll lll --'. .::£ lll lll ru w ru.., L L lll lll Q) lll Q) z >-c 0 4-Q) ·-Q) a:: -o Q) ilJ"U -ltiO..COitl ..... Cl) U'l ~ ..... lll-E-1- Cl) Cil ro o z 11~'{~ 1. Paint fire hydrants. 2. Operate hose clamp. 3. Operate 1ife net. ..3 4. Prime engine pump. 5 5. Straighten up quarters in station. s c 4. The imaginary rater we discussed above has completed the first three ratings on all 204 check list items and has returned to make Importanceratings for the first five tasks. Since firefighters in his department do not paint hydrants, he did not rate Task 1 for importance. Hesaid that clamping hoses was a 1ittle more important than most of theother tasks. Operating life nets was rated as about as important as other tasks, since, in most cases, other safer means of evacuating people are available. Pump priming was rated as clearly more importantthan the other tasks, since the rater's department uses only centrifugalpumps, and so they must be primed before any fire can be extinguished. Finally, straightening up quarters, although it takes a good deal of time, was rated as relatively unimportant compared with the other tasks1i sted. In Summary First, decide whether your firefighters perform the task at al 1, thenwhether they perform it on a regular or on a relief basis, and then what proportion of their time they spend doing it. Do this for a 11 204 tasks. Then go back and decide the importance ofeach task that is performed~firefighters in your department. Cautions 1. Rate only what firefighters do (not trainees, not officers, and notequipment operators if they are in a separate job classification~ 2. Rate only the tasks firefighters in your department actuallyperform, not what you wish they did. 3. Rate only whether they perform each task, not how well they perform it. 4. When you estimate relative time spent and relative importance for each task, consider all 204 tasks, not just the other tasks in thesame task grouping. -- 5. Try to use all the columns on both the Time Spent and the Importance scales. Do not I imit your ratings to the middle or the high rangesof the rating-5cales. c 5. Rater and Department Information Please write: 1. Your name 2. Your rank or position title 3. Your city 4. Total number of uniformed personnel in your department (if known) Use an X to indicate: 5. Are firefighters in your city: paid volunteer both 6. Which of the following types of fires is your department called upon. to extinguish? a. Residential b. School c. Large factory and manufacturing (more than 50 employees) d. Small manufacturing (50 or fewer employees) e. Large mercantile (department stores with three or more floors or more than 5,000 square feet of floor area) f. Small mercantile (small shops and stores with g. Apartments or hotels with one or two floors h. Apartments or hotels with three to ten floors i. High rise apartments or hotels with more than , j. Office buildings with one or two floors k. Office buildings with three to ten floors 1. High rise office buildings with more than ten m. Hospital or nursing home n. Light, one-or two-engine private aircraft o. Common carrier passenger aircraft p. Military aircraft q. Grass, scrub, or open field r. Automobile 7. Which of the following flammable or hazardous materials in your department's jurisdiction? s. Liquid bulk or liquid pressurized gas storage t. Flammable metals u. Radioactive materials v. Volatile gases and vapors one or two floors) ten floors floors are present T'me Spentlllmportance 1 -"'"' -"' "' "' "' .3 ~ "' .... .... "'.... ...."' "' "' OJ .J::. ... .:;,:, .... ~ OJ OJ ..., c Ill c: ...... ..c. 11'1 "'.c. 0 I'U ttl 1'0 0 4-J..::f. J,.4J u u ..... 0 \II J't Ill 0 C L L. C I'D o L o ro "'-""' 0 .... Ill""" 0 ..c. Q. ClJ a. ..c ... "' u .... u .... ~ -~ 5 .§ ... "'0 Q) Q) ""0 ..... 0 uE Q) .J::. L .C QJ c 11'1 cu c .... "-...... Q) ..... 1... ro lfl..., '-I'D0 ~ ttJ 0 ..C :J IU ,..... -a. ...... a. ~ ~ 0 ~ ~ ' 0 E 0 5 g~o~~ o.. ClJ ClJ a "'a. u ..c Ill ..c u e~ 111-e ~ ..., ttl ..... ·-~ ttJ :::: · ' "' -"' ·-- \II QJ cu Ol 11'1 ·-Q) ·-Q) TASK "' -I'U VIE L. L. 111-E-' " ClJ E cu ttt 0 I'D Cli tQ. 0~ a: l.n...JVl;L._.I -.J<(Vl<(:C I 2 3 4 5 l 2 ~ 4 5 INSPECTING, TESTING, CLEANING, MAINTAINING, AND READYING VEHICLES I. Start apparatus to check engine and pump, ladder, or other systems for proper operation. 2. Clean, wax, and pol ish fire apparatus and remove dirt, mud, grease, oil, and soot, to maintain appearance and inspect for damage. J. Check levels of fuel, oil, transmission fluid, radiator fluid,· booster tank water, battery fluid, etc.,and fill as needed 4. Perform periodic test of pumping capacity and pressure or of aerial ·ladder structural strength. 5. Clean, test, inspect, and refuel mounted electrical generators. INSPECTING, TESTING, CLEANI·NG, MAINTAINING, AND READYING EQUIPMENT ASSIGNED TO APPARATUS 6.. Clean, dry. inspect, and properly secure stretchers, litters, and resuscitator equipment, and replaceused first aid supplies and equipment. ]. Clean, dry, inspect, and properly secure rescue equipment such ·as K-12 saws, spreaders, pneumatic air drill, etc. 8. Clean, dry, inspect, and properly secure firefighting and salvage equipment assigned to apparatus(ladders, hand tools, portable breathing equipment, ropes and tackle, ventilation fans, salvage covers and runners; hose appliances, nozzles, and extra hose couplings; leather straps and belts; coat, hat, and boots; etc.). 9. Clean, dry, inspect, and replenish oxygen or compressed air tanks. 10. Clean, Inspect, refuel, and properly secure engine powered equipment such as chain saws, nonmounted portable generators, etc. n INSPECTING, TESTING, CLEANING, MAINTAINING, AND READYING COUPLINGS ANO NOZZLES CT\ 11. Wash, dry, inspect, and rack hoses and coup! ings for future use. "T:ime Spent lflinportancej TASK E .... ~ .... Q) a. .... Q) > ~ >c 0 .... Q) ·-Q) "' "' "' '"' ... Q) Q) .!: "' .n.<:: ... ..,. _,. ... 0 ..... ~ U"' 0.. _,."'0 ... \It ..... 0... "' ... .......... "0 Q) Q)"O QJ ..c L.. ..c Q) L.. ... QJ 4J 'ftJ 0 ...J:: 0 rtl a. ... a. E c: 0 c: E 0 .. "' 0u ...c C/'1 .r=. u........ Q) -U't QJ GJ 0'1 fl] 1/) E L I... E iU ta 0 rt1 Vl.....JVl:L-.J 1 7 ~ 4 5 "' "' ..(. _,. "' "' "' "' ~ ..., ... ... "' "' Q) ..:; 4..1 .:J£ w .£. ... c "' c ... 0 I'U ttl 111 0... ...... c "'c: "' 0 ... 0 "' .<:: a. Q) a..c '"'E.c E ..., ·..... ·... 0 ... c "' Q) c n:J Ill~ L. I'U ~! ~ 2 ~ 0 E 0 o. GJ OJ a E-111-E ·..... ttl ..... ·... ... VI •-QJ •-QJ 11'1-E-'-Q) "' 0.....J ~ >-c 0 ..... Ill ·-QJ a:: "' "'.... .... "' Ill..C VI V1.£. ..... ..::s. .;L. ........ 0 111 Ill L/'1 0 "'-"'"'0....,1/l....,O ..., "' ...,.... ..., .... "0 QJ QJ ""0 Q) ..c L. ..c QJ 1... +-' QJ +-' L ttl o ..c: o rc a. ..., a. 5 ~ 0 ~ 5 U ..C VI ..C U ........... -Ill ,...;.. 11'1 CLI Q) O'l rn VI E ''E ~ >~ 0 ""' ·"' a:: .... .... "" "' Q) V). 11) .... '.c.~-'""~,£ Q,) cu ..., c: !J) c <1-J ..c. !J) "" ~ o ro ~ ~ o ..... ..;,{. ..::L..... ..... .......... 0 "" "' !J) 0 c ~ 'c tO ..::L ra m o 'o ro 0 ..... V'l...., 0 £. 0. Q) CL..C ....., tO .._, ....., E ..r: E...., lr... ._, L •<1-J •"'0 Q.l Q) "'U ..... 0 <1-J Q) ..S:::. L .£. Q) C !J) Q) C '..... Q) ..... \... "'!J)....., '-t'J ~0"50~ ~~~g~ ECOCEO E 0 o ro I'D o c.. Q,) Q) a. U.L:111..CU E-11)-E ..... tO....., ·..... ro ..... ·-Q) 4-J ..... -1/) Q) Q) Ol V'l. ·-Q,) ·-QJ l'tl V'l E ''ll'l-E-'-E cu ro o ro v rc· o .~ -J Vl ::t: _..J .....J -nJ o ..c o rc .... OJ V'l. 0 ....... ..... Q. ~ Q. ... L-0 E 0L c E c 0 c E 0 E Q. 0 o 1t1 ro o a. QJ Q) 0 "' u ..c cil ..c u E-V'l-E TASK ... ..... ~ "'~ ·-::: r:J ~ · -QJ > ·-"' -V'l (I) Q) 01 V'I ·-QJ ·-Q)"' -rt1 Ill E L.. 1... 11'1-E-L_'! E OJ ttl o ru V)__J(,I"):£:-.J "' "' 0 -l-ftiii'IE'-'-IJ)-E-'-Q) cu E GJ ra o ra QJ ro oz ex:: t/)_.JI.fl:t:-.J -lctVl<(l: I 1234512345 Climb, work from, and descend ladders carrying people or equipment, and using appropriate safety equip 64. I , · ment or procedures. OP£RATING AERIAL EQUIPMENT 65. Stabilize apparatus using wheel chocks, stabilizing pads, and stabilizing jacks or outriggers. Elevate, rotate, and extend aerial ladder bed and fly sections for suppor-ted or unsupported operation. 66. 67. Operate snorkel or elevating platform from ground controls or platform controls. 68. Mount and/or operate master stream appliances from aerial ladder, snorkel, or elevating platform. 69. Operate elevating master device (Squirt) EXTINGUISHING FIRES 70. Don and operate portable breathing equipment. 71. Operate any class of fire extinguishers. 72. Operate hose 1lnes with stream or fog nozzles and with master appliances. 73. Operate hose 1ines with foam generators and proportioners. 74. ~th~rt portable sprinkler heads through roofs of bui ldlngs to extinguish attic or loft fires. 75. Operate Installed sprinkler systems. 76. Locate and operate shut-off valves for gas, electricity, oil, and water. 77. Beat grass fires with broom, paddle, or shovel. 78. Operate portable, hand operated pump to extlrigui;h grass or scrub fires. I I 79. Operate atti~and cellar pip~---__ Time Spent Importance "' "' ~ ~ ~"' ..,"'Ill .... .... .... .... "' "' .c.~~"' "' "'"'J::. ...... c: lA c: +-1"' ..c::"' U1..C"' o ru rc ro o ..,"' -"' -"'.., .., .., .., 0 Vl 11'1 Ill 0 c .... .... c Ill~"' i'lJ 0 '-0 (I] .. 0 +-1 Vl olj 0 ..c c.. G> a. .c .., .... ..,"' .... .., "'"'E ...c E"'"' ·-..... ·-.., -o "' .... 0 E d) .c I... ..c."'"'Q) c "' "' c .... '-.... QJ....., '-1'0 11'1 +-~ L f'J >-., 0 .c 0 ttl.... c E c 0 c E 0 E 0 .2 a. .., a. ~ ~ 0 ~ ~ tU VI E '-'-r.n-E-'-E IU t0 0 ro -"-~ "' "' ~ ~ '-'- Q) "' Q) +.I~ "' "'L. .r::. +.IL '- Q) Ill ...., c 1.1'1 c +.1 ..c 11'1 V'l.£. oro ro ro o .., ..., .., .u..:::L. ..::L..u 0 Vl t/'1 1/1 0 c~ '-orocl n _ roo' "'-"-.. 0 0 ..... 1/1 ..... .., .., .. z -~·5 .§~ w '-.., ' .., 0 .., "'0 QJ Q.)~ ~ (1) ..C L ...C Q) c U"' QJ c L +.I QJ .U L ro V'l .u L ttl 0 >-II ro 0 L 0 ro ' a. ..., a. ~~ cg g ~ '";: c: E c 0 c E 0 E 0 a... Q) QJ 0. 0 o ro ro o "' a. L) ...c Vl ....c:: u E-\/)-E ·-+-~ ro ......, · '.., "'..., .., .., -ro 1/) E '-Ill ~~~-'-1.1'1-E-' TASK cu E