us ))(Of. ~ojG: DOCUrt.ENTS & M!CRQFORMS DEPT. I.OCK'.VOCD l.i[fi;~·Riiil LIBRAR'f STATE UfliVERSITY of NE'.'I YO!lll at BUFFALO BUFFALO, hE\'/ YO~i( 142CO 7n 12 '- Q) ..cE Q)0 c Q) .r:.UJ 0+-' L. 0 ctS :J Q)-o (/J 0 Q) L. a: a. CIJ Q)· +-' ... 0 ::J :J ... ·-... C/) c: -c 0 -'= '- L. 0 a.. a:s Q) C/) ..c II) Q) co 0 (J) a: ,... >a ~ .. a1 Q) E L. (]) .c < E CIJ Q) en• (]) (,) . :J a: c Q) F='oreword The U.S. Army Research Institute (ARI) is a field operating agency of the Deputy Chief of Staff for Personnel. An applied research organization headquartered in Alexandria, Virginia, ARI carries out its research through three laboratories and an Office of Basic Research. Its Manpower and Personnel, Systems, and Training Research Laboratories in turn serve as headquarters for research groups located in Alexandria and at major Army installations. Nine scientific coordination offices at FORSCOM, TRADOC, USAREUR, Fort Huachuca, Boise, ADEA, OTEA, and in Japan and London provide a continuing identification of Army requirements, technology opportunities, and transfer of ARI products. ARI executes a core program of research and develops products that affect every Army mission that has human performance as a component. • The Manpower and Personnel Research Laboratory focuses on designing and developing methods to improve the process of recruiting, selecting, assigning, and retaining Army personnel. • The Systems Research Laboratory program aims at ensuring that the capabilities of soldier operators and maintainers are considered in the design of new weapons and support systems. • Drawing on existing theories of learning, motivation, cognition, information processing, and measurement and evaluation, the Training Research Laboratory continuously examines and refines the Army's training methods. • The Office of Basic Research plans and executes (primarily through contracts with the academic community) long-range research in four principal areas: planning, problem solving, and decision making; unit performance; skill building technologies; and designing systems for people. Thus, ARI conducts people-related basic research and exploratory, advanced, and engineering development, which lead to products based upon a scientific understanding of soldiers and the tasks they perform. ARI research helps the Army to recruit and retain the best people .and to. maximize their individual and organizational effectiveness in light of modern battlefield tactics. By applying state-of-the-art concepts and methods of social and behavioral science, ARI helps solve current Army problems and helps make the future more manageable. i ARI's research falls into five general program areas: • Structure and Equip the Force • Man the Force • Train the Force • Develop Units • Maintain Force Readiness This book describes recently produced products that are representative of ARI's achievement. Listed by program area, these products have either been adopted by their prospective users or are ready for fielding. The contents of this book will be updated periodically as other ARI research projects reach fruition. Questions and comments about this document.may be addressed to: US Army Research Institute for the Behavioral and Social Sciences 5001 Eisenhower Avenue, Alexandria, Virginia 22333-5600 Attention: Plans, Programs, and Operations (PERI-PO) Phone: (202) 274-8840 or Autovon 284-8840 ii ARI Research Products Structure and Equip the Force HARDMAN Comparability Analysis Methodology Guide 3 LHX (Light Helicopter Experimental) MANPRINT Support 4 Questionnaire Construction Manual 5 Human Factors Evaluations in User Testing During FY 85 6 Tactical Intelligence Analysis (Training Circular) 8 Realistic Air Defense Engagement System (RADES) 9 Reverse Engineering 10 Computerized Intelligence Aids: ENCOA (Enemy Courses of Action) 11 Recommendations for TACFIRE Improvements 12 Man the Force Enhanced Measures of On-the-Job Performance (MOS-specific) 15 Measures of General Soldiering Effectiveness (Army-wide) 16 Computerized Selection and Classification Measures 17 Soldier Quality Task Force 18 Army Manpower Cost Modeling System 19 Recruit Quality and Soldier Performance Model 20 Multivariate Enlistment Model 21 Multivariate Reenlistment Model 22 Delayed Entry Program (DEP) Management Model 23 Computerized Adaptive Screening Test (CAST) 24 Data on Why People Join the Army 25 New Armed Services vocational Aptitude Battery (ASVAB) Composites 26 A Profile of Army Families in USAREUR 27 Train the Force Multipurpose Arcade Combat Simulator (MACS) 31 Combat Vehicle Identification (CVI) 32 Training Requirements for NBC and the Future Integrated Battlefield 33 iii ARI Research Products (Continued) Train the Force (Continued) Develop Units Basic Skills Resource Center (BSRC) 34 Device Effectiveness Forecasting Procedure (DEFT) 35 Continuous Operations Guidelines 36 Design for MOS 19K Basic Noncommissioned Officer Course 37 Army Job-Relevant Basic Skills Education Program (BSEP) II Curriculum 38 Transitional Performance Aid (TPA) 39 Bradley Infantry Fighting Vehicle Procedural Guides 40 Bradley Infantry Fighting Vehicle (BIFV) Ammunition Storage and Reload Systems 41 Grenade Launcher Range-Estimating Device for the M203 42 Training Materials for the Rapid Train-Up, Cross- Training, and Sustainment of M60A3 Armor Crewmen 43 Unit Rifle-Marksmanship Training Guide 44 Skill Retention Model 45 Unit Fire Direction Center Training Program for the 81mm Mortar 46 Generic Recommendations for Improving Civilian Performance Appraisal Systems 49 Leadership Lessons Learned at the National Training Center 50 Leadership Improvement Modules 51 Formative Evaluations of Corpus Christi Army Depot Sociotechnical Systems Program 52 Advanced Technology Unit Training and Management System (ATUTMS) 53 Stratified Systems Theory for Executive Leadership 54 Lessons Learned During 7ID(L) Transition to Light Division Status 55 Electronic Clipboard 56 iv ARI Research Products (Continued) Develop Units (Continued) Program of Instruction (POI) for Individual Ready Reserve (IRR) Aviator Refresher Training 57 Model for Guiding Implementation, Evaluation, and Integration Support of New Training Products in the Army 58 Maintain Force Readiness Computerized Hand-Held Instructional Prototype (CHIP) 61 v Structure and Equip ,the Force AREAS - PRODUCT: HARDMAN Comparability Analysis Methodology Guide Army Problem/Need/Issue: A system is needed that provides decision makers with the types of key human resources information necessary for making sound decisions early in a weapons development process. ARI Solution: The Navy's HARDMAN (hardware vs. manpower) Comparability Methodology (HCM) is a structured approach for determining the Manpower, Personnel, and Training (MPT) requirements of a weapon system in the earliest phases of its development. HCM was first evaluated and applied to the Army's Corps Support Weapons System (CSWS), Division Support Weapons System (DSWS), Single Channel Ground/Airborne Radio System (SINCGARS), and Remotely Piloted vehicle (RPV). Then, a detailed five-volume HARDMAN Comparability Analysis Methodology Guide was developed for Army use. The guide specifies in greater detail how to conduct HCM and thereby expands the base of contract and in-house personnel able to perform such analyses. Applications/Results: • Application of HCM to all Army major systems was directed by the VCSA• Soldier Support Center -National Capitol Region was made as the proponent for HARDMAN analyses • • Applications of HCM to the following systems have been completed or are underway: • All Source Analysis System (ASAS) Armored Gun System (AGS) Army Hawk (PIP-III) Air Defense System Elevated Target Acquisition System (ETAS) ...TIM I lia;7'/~ _. -CUIU'IINT nAIJ\1111110 EJDT C':"" - COfifCI" 1 I I ~-!t/. ..:=:::au ..~:l ai""'T Close Combat LASAR Countermeasure System (STINGRAY) Light Air Defense System (LADS) \~. / 11 Light Helicopter Experimental System (LHX) Patriot Air Defense System ~~re·1/ffi SGT York Air Defense System The following HCM applications are planned: U'UUUU ~ 1 -tA.• Advanced Field Arb llery Tactjcal Data System (AFATADS) ~ '-. ~} • 001 0 Long Range Anti-Tank System (LRAT) ~ / /' Forward Area Armored Logistics System (FAALS) Proponent: u.s. Army Soldier Support Center, National Capitol Region DecembP.r 85 3 PRODUCT: LHX (Light Helicopter Experimental) MANPRINT Support Army Problem/Need/Issue: The Army needs to focus and integrate manpower, personnel, training, human factors engineering, health hazard assessment and systems safety in the weapons system acquisition process. Manpower and Personnel Integration (MANPRINT) is an umbrella concept that integrates these six areas and allows for early identification of high resource drivers so that curbs can be placed on "people" costs and skill demands. ARI Solution: The approach to this effort is to serve as a MANPRINT integrator/technical advisor to TRADOC Systems Managers and AMC Program Managers for major Army weapons systems, and to the ODCSPER for selected weapons systems. This includes development of an overall MANPRINT management plan for the systems manager; review of HARDMAN-type analyses; refinement of existing or development of new manpower, personnel and training (MPT) analyses; and support foe the MPT portion of the Human Factors Engineering Analysis (HFEA) document. Applications/Results: Efforts for the LHX Integrated Logistics Support (ILS) manager have served as the prototype for MANPRINT support. Actions taken include the following: • An LHX MANPRINT management plan has been developed that reviews the HMPT issues and how they shouia be addressed in the development cycle. • Contractual support has been initiated to investigate Reliability, Availability, and Maintainability (RAM) issues as an organizational impact, using AMORE and HARDMAN methodologies. • A significant contribution to the activities of the T800-XX-800 Engine Source Selection Evaluation Board has also been made. This was the first major Army procurement that had RAM and ILS elevated to to Area status and weighted equally with the Cost and Technical Areas. • A project is underway to provide prime contractor candidates for LHX with feedback on their MANPRINT strengths and weaknesses. This is part of an ef fort to strengthen their MANPRINT performance. Other Candidate systems for the Group's participation include AFAS, ACCS, Stingray, AAWS-M, JTACMS, and ASAS. Proponents: ODCSPER; TRADOC; AMC December 85 4 PRODUCT: Questionnaire Construction Manual needs a manual for the use and guidance of those who are tasked to Army Problem/Need/Issue: The Army develop andjor administer questionnaires as part of Army field tests and evaluations, such as those the Combat Developments Experimentation conducted by the TRADOC Combined Arms Test Activity (TCATA), Command (CDEC), the Operational Test and Evaluation Agency (OTEA), and the several Army Boards and Schools. the research literature on the construction and administration of ARI Solution: In 1975, ARI reviewed two products. One was the forerunner of this volume. It questionnaires and interviews, and produced It was revised and is was titled "Questionnaire Construction Manual" and was published by ARI in 1976. sued in quantity in 1977 as ARI special publication P-77-1. The other product, issued in 1977 as P-77-2, Bibliography." was "Questionnaire Construction Manual Annex: Literature Survey and point where the first review had ended in 1976. An In 1983 the literature was again reviewed from the analysis of this latter literature survey provided the basis for this current manual. Applications/Results: • The manual covers the following main topics: guidance relating to the preparation of items for questionnaires and for their Information and assembly and arrangement into a complete questionnaire The importance of, and procedures for, pretesting questionnaires prior to their regular administration Characteristics of respondents that influence questionnaire results The analysis and evaluation of questionnaire responses means of an interviewA number of considerations regarding the presentation of questions by • The general content and concepts of this manual are applicable to more than operational test situations. Thus, it should prove useful to all individuals involved in the construction and administration of surveys, interviews, or questionnaires. • A detailed report of the literature survey including abstracts and a bibliography is being pub lished as a companion volume and is titled "Questionnaires: Literature Survey and Bibliography." Proponents: TRADOC, TCATA, OTEA 5 December 85 PRODUCT: Human Factors Evaluations in User Testing During FY 85 Army Problem/Need/Issue: Developmental testing, operational testing, and reports from the field have emphatically revealed that the Army's Materiel Acquisition Process traditionally gives inadequate attention to the characteristics of operators and maintainers of new Army systems. Therefore, there is a continuous requirement to conduct human factors evaluations of new systems in order to identify soldierrelated deficiencies before the systems are fielded. ARI Solution: In conjunction with OTEA, TCATA, and the various test boards, ARI conducts human factors evaluations of Army systems in operational environments. In accordance with the new "Continuous and Comprehensive Evaluation" concept, human factors evaluations are now initiated early in the acquisition process in order to identify soldierjsystem interface problems while they are relatively inexpensive to correct. Applications/Results: The ARI Field Unit at Fort Hood sponsored or conducted human factors evaluations of several Army systemsduring FY 85: • Armored Maintenance Vehicle (AMV) Concept Evaluation: Two prototypes were examined and found to reduce down time for maintenance operations. No vehicle safety hazards were identified. To date a vehicle type has not been chosen. • High Mobility Multipurpose Wheeled Vehicle--Heavy variant (HMMWV-HV): The human factors and safety evaluation of this system was in the planning phase during FY 85. • Quiet Reliable Generators (QRG): Twenty-four electrical generators were evaluated. Numerous human factors and safety problems were uncovered. Most predominant were problems related to the locations of fuel points, check points, and controls. Findings provided input to selection of a number of generators for interim use. • Multiple Subscriber Equipment (MSE): This equipment provides voice and data communications between division level and multiple field users over network radio. Two systems were evaluated during demonstrations by their manufacturers. Over 100 human factors/safety-oriented problems were noted and recommendations for production improvements were made. December 85 6 • M9 Armored Combat Earthmover (ACE): During the follow-on evaluation of the ACE, 40 human factors deficiencies were noted, including high noise levels, high cab air temperatures, high vehicle vi bration, uns~fe hatch design, and unsafe blade folding procedure. • NAVSTAR Global Positioning System: Two candidate systems were evaluated. The better system was selected and guidance for system improvement was provided to the developing contractor. • Self-Propelled Howitzer (M109E4): It was determined that soldier training was effective and that prior product improvements had eliminated significant hazards and interface problems. • SGT York (Division Air Defense Gun--DIVAD): Evaluation of this weapon system identified human factors problems of varying degrees of severity. Recommendations for minimizing their degradative impact on system performance were provided. • SINCGARS-V: This replacement for the Army's current VRC-12 and PRC-77 radios was determined to have significant human factors interface problems. The impact of each problem was estimated and potential solutions were presented for use by the project manager in product improvement efforts. • Single-Subscriber Terminal (SST) (AN/UGC137A[V)2): Despite successful performance in initial opera tional test and evaluation, 37 human factors problems were identified. Corrective solutions for software, hardware, and workstation design were suggested. • Tactical Communications, Command and Control System (TC3S): The AN/TYQ30, 31, and 32 were evaluted for user requirements. Needed system improvements were documented. "o M1 ABRAMS TANK • TRAILBLAZER o BATTERY COMPUTER SYSTEMProponents: u.s. Army Infantry School (HMMWV); u.s. Army o FIREFINDER AN/TP0-37 RADAR o JOINT TACTICAL FUSION TEST Engineer School (QRG, ACE); MICOM (DIVAD); BED IBETAI o SINGGARS CECOM (MSE, NAVSTAR, SINCGARS-V, SST, TC3S) 7 December 85 PRODUCT: Tactical Intelligence Analysis (Training Circular) Army Problem/Need/Issue: The Army needs a way to train, in the classroom, the "soft skills" pos sessed by experienced intelligence analysts. Previously, the school concentrated on teaching intelli gence gathering, posting, and cataloging procedures, but many years of field experience were required for an analyst to be able to successfully combine data from a number of intelligence sources and make a judgment about enemy intentions that would be of use to a commander in the field. ARI Solution: Two hundred analysts with field experience were interviewed, and a model of all-source analysis was developed. Based on the model, a book was developed entitled "An Introduction to Tactical Intelligence Analysis: Cognitive Preparation for the Battlefield for New and Experienced Analysts." Among the topics presented are: the nature of the intelligence analyst's job; the working environment of the tactical intelligence officer; aspects of threat; techniques to aid in understanding threat; management skills; thinking skills; cognitive aids; and special topics for new and experienced analysts. Applications/Results: • The US Army Intelligence Center and School (USAICS) is publishing the Introduction to Training Intelligence Analysis as a training circular. • A draft version of the circular has been in use at the USAICS for more than a year as part of class materials for new in telligence analysts. • The training circular, in addition to being part of the school curriculum, will be distributed to the field, particularly to reserve units for the purpose of maintaining currency of skills. " u.,.hot •t tiR~a 1 ,,, . :.:..,. ......ttot at ..... 2 • The USAICS has developed an automated version of some of the ~/ ,!"!""' t ...."',":,~:rr......~·;;n..ahot •t outcow.a materials called. "How to Analyze." • The USAICS has incorporated materials on "Nine Thinking Skills" " ..;~~~=::~~~;-~~~~~::.~.~-~-~-~-:_:,;);;;;;;;;;;;:~ and "Six Analytical Tasks" into FM 34-1, the primary reference document for intelligence analysts. Proponent: u.s. Army Intelligence Center and School December 85 8 PRODUCT: Realistic Air Defense Engagement System (RADES) Army Problem/Need/Issue: The Army needs to increase the effectiveness of man-portable and shortrange air defense (MANPAD and SHORAD) weapons systems by improving the performance of the personnel using these systems. Need has been intensified by decision to terminate the SGT York program. a portable, outdoor facility using scale-model remote-controlled ARI Solution: ARI has developed RADES, aircraft and simulated firing techniques. This system is a means of testing, assessing, and training environment. As a test bed, RADES individuals and crews in a realistic Forward Area Air Defense (FAAD) has been used to measure crew performance under different environmental conditions and against various STINGERs, REDEYEs, VULCANs, and CHAPARRALs and alternatives aircraft tactics, maneuvers, and patterns. to SGT York are being used with this system. Applications/Results: • Valid combat readiness performance measures • Ability to identify aspects of gunner's job that could be improved through training or the use of performance aids • Data being used by Air Defense School to evaluate alternatives to Sgt York DoD to obtain human performance parameters for SHORAD combat models -~ DoD to obtain requirements for man-in-the-loop SHORAD simulator • Products of RADES research include: Fixed wing aerial targets and helicopter targets ~ REALISTIC AIR DEFENSE ENGAGEMENT SYSTEM Target infrared sources and hit indicators State-of-the-art miniature radio frequency tRADES) transponders RADES universal junction box and engagement soft ware for FAAD weapon systems RADES test and measurement bed Generic FAAD fire unit trainer Proponent: u.s. Army Air Defense Center, u.s. Army Air Defense Artillery School 9 December 85 PRODUCT: Reverse Engineering Army Problem/Need/Issue: As the Army modernizes, it must ensure that its new equipment is suitable for the available manpower. Materiel systems are not performing as desired because these systems have not been designed with adequate consideration of the performance capabilities and limitations of their operators, maintainers, and repairers. ARI Solution: The Reverse Engineering process ¥ras developed and, to date, has involved the examination of the STINGER, Multiple Launch Rocket System, BLACK HAWK Helicopter, and the Fault Detection and Isolation Subsystem of the M1 Tank. By using documentation and data on each weapons system, this examination revealed how human factors, manpower, personnel, and training (HMPT) issues were addressed at critical points in the weapons systems acquisition process (WSAP). The effects of these actions on the subsequent weapons systems' performance were also integrated into the results, and recommendations for improvements in the WSAP were presented to the Vice Chief of Staff of the Army, GEN Maxwell R. Thurman. Applications/Results: • Improvements to two key documents (Request for Pro posal and Required Operational Capability) in the materiel acquisition process have been developed. They ensure that human performance requirements are addressed adequately and that systems are designed to achieve acceptable performance when operated and maintained in the field by soldiers possessing the range of expected capabilities. -a. M1TANK • Improved weapons system performance as a result of considering both man and machine. • More effective use of the Army's manpower pool. Proponent: Vice Chief of Staff, Department of the Army " December 84 10 PRODUCT: Computerized Intelligence Aids: ENCOA (Enemy Courses of Action) Army Problem/Need/Issue: The Army must improve the prediction of enemy courses of action by increasing the capability of intelligence analysts. ARI Solution: ARI has developed intelligence aids that structure the process of both organizing and evaluating information and integrating it into a final judgment. ENCOA aids break down the decision problem (predicting enemy mission accomplishment) into broad factor categories and increasingly spe cific individual component factors. The factor categories are terrain, u.s. forces, enemy forces, weather, and risk. These five categories are further divided into 25 individual component factors. ENCOA is used best before battle when ample time is available to assess of battle, terrain, and enemy factors. Additional aids can be used to judgment inherent in Intelligence Preparation of the Battlefield (IPB). Applications/Results: • ENCOA is generally applicable for any tac tical intelligence environment in which analysis must evaluate alternative enemy courses of action or avenues of approach. • ENCOA is specifically applicable for devel oping situational and event templates when using IPB. • ENCOA will be used for stand-alone training by students learning IPB. • ENCOA will be used in automated training exercises for basic and advanced courses at the u.s. Army Intelligence Center and School. Proponent: u.s. Army Intelligence Center and School carefully all relevant order support other areas of human December 84 11 PRODUCT: Recommendations for TACFIRE Improvements Army Problem/Need/Issue: ARI, in its February 1981 Research Product 81-26, "Human Factors Analyses of User/Operator Transactions with TACFIRE--The Tactical Fire Direction System," noted that the TACFIRE soldier/machine interface (SMI) design deficiencies of greatest significance were as follows: • Lack of a command language, limited use of menus, and overly complex design of function keys for data input. • Inconsistent format and density of alphanumeric displays. • variation in keyboard configurations at different work stations. • A plethora of TACFIRE message formats, data element names, abbreviations, and codes whose sheer volume, inconsistent design, and use require ten volumes of user manuals. None of the material in the manuals is available on-line. ARI concluded that this might be regarded as a major problem, because it imposes human memorization requirements well beyond normal limitations. ARI Solution: Recommendations to improve the TACFIRE operator/machine interface were made in the fol lowing areas: control methods, display format, data entry assistance, message composition aids, data retrieval assistance, glossaries, and error handling. Each recommendation included an assessment of its impact on user/operator performance if implemented. Applications/Results: • At least 28 of the ARI recommendations were applied by Litton Systems Inc. to the design of the soldierjmachine CONTROL METHOOS interface (SMI) of LTACFIRE (Lightweight TACFIRE). DISPLAY FORMATS • LTACFIRE is currently undergoing testing by ADEA at DATA ENTRY ,. Fort Lewis, Washington to assess its ability to further increase the effectiveness of field artillery command and control operations. • If the ADEA tests substantiate their value in improving soldier/system performance, the ARI design recommenda tions will serve as the model for the next generation TACFIRE SMI. Proponent: CACDA December 85 12 Man the Force 1 3 PRODUCT: Enhanced Measures of On-the-Job Performance (MOS-specific) Army Problem/Need/Issue: The Army needs a job-based selection and classification system to meet congressional and Department of Defense requirements for measures that are linked to job performance (vs. training performance), and to ensure that the right people are assigned to the right jobs. Existing measures of MOS performance (e.g., the SQT) are primarily written tests that do not reflect all of the dimensions of on-the-job performance, and therefore are insufficient criteria for the validation of current and experimental selection and classification measures. ARI Solution: ARI has developed multidimensional MOS-specific measures of on-the-job performance in cluding hands-on performance, knowledge tests, supervisor rating scales, and peer rating scales. A full set of training and on-the-job (OJT) measures now exists for 9 MOS, and a set of school-based training measures exists for an additional 10 MOS. Correlative data derived from the first 9 MOS will be applied to the other 10 to yield OJT-based prediction capability. Applications/Results: • Measures are currently being used in a concurrent validation of experimental predictor measures. After the set of predictor measures is refined, the same performance measures will be administered in a longi tudinal validation of the revised predictors. • Measures have been provided to the proponent schools (Infantry and Signal, e.g.) for possible future use in their tes,ting programs. • Information on the development of the performance measures has been provided to the Individual Train ing and Evaluation Directorate. Proponent: DCSPER, DA December 85 15 PRODUCT: Measures of General Soldiering Effectiveness (Army-wide) Army Problem/Need/Issue: The Army needs a job-based selection and classification system to meet congressional and Department of Defense requirements and to ensure that Army manpower needs are met by the selection of qualified applicants. Comprehensive measurement of job performance includes Army-wide indicators of general soldiering effectiveness. The Army currently does not routinely collect general soldier performance data on first-tour soldiers. ARI Solution: ARI has developed and, following extensive field testing, refined a set of behaviorally anchored rating scales for evaluating performance along a number of dimensions of Army-wide soldier characteristics (e.g., leadership, discipline, NCO potential). RATINGS Applications/Results: Supervlaor PHr • Army-wide rating scales are currently being used in conjunction with MOS-specific performance measures in a concurrent validation of experimental predictor measures. After the set of predictor measures is refined, the same rating scales will be administered in a longitudinal validation of the revised predictors. • Measures have been briefed to the Evaluation Systems SeH Office, u.s. Army Personnel Center, for possible fu ture application to the Enlisted Evaluation Report (EER) system. Second-tour measures, to be developed, will also be made available for use in the EER system. Proponent: DCSPER, DA 16 PRODUCT: Computerized Selection and Classification Measures Army Problem/Need/Issue: The Department of Defense directed the Army to procure and field a computer adaptive testing (CAT) system that yields results equivalent to the present paper-and-pencil (P&P) version of the Armed Services Vocational Aptitude Battery (ASVAB) and that has the flexibility to include However, current Army selection new and innovative tests now being developed by the four services. and classification standards are not linked to computer-generated test scores. Consequently, the Army is concerned about the equivalence of the present P&P-ASVAB test results to CAT-ASVAB results. Also of concern is the expansion capability of the CAT system to be installed in the u.s. Military Entrance Processing Command (USMEPCOM). ARI research and development is oriented toward assessing the expansion/flexibility of ARI Solution: This includes evaluation of modifications and any proposed CAT system to include new types of tests. additions to any proposed CAT system, research on the relation among ASVAB scores, training scores, and scores from new test material, and research on the computer adaptability of new test types. Applications/Results: • Data collection is nearly complete on CAT-ASVAB and P&P-ASVAB vs. training measures at Fort Belvoir, Virginia. • Expansion of the CAT system is underway, to include Project A tests (job performance-related selection and classification measures). • This project will provide ODCSPER with critical information upon which to make a decision for the purchase of a CAT system for service-wide accessions. Proponent: DCSPER, DA 17 December 85 PRODUCT: Soldier Quality Task Force Army Problem/Need/Issue: Prepare report with and for DCSPER on the Army's soldier quality requirements (in terms of AFQT category and high school diploma status); include in the report qualitative and quantitative justifications for quality requirements. ARI Solution: A task force was organized made of scientists from each ARI laboratory, DCSPER, SSC-NCR (as well as CAA for planning) under the direction of Dr. Lawrence Hanser (Selection and Classification Technical Area). The task force prepared operational definitions of the issues and questions, prepared a research plan to address the tasking, collected soldier quality and performance data, analyzed the data, and wrote the draft report. In addition, the plan and report were briefed to ARI HQ, DCSPER, and other relevant Army agencies. Applications/Results: The report was well received by DCSPER and forwarded to DoD and Congress. The Army's soldier quality requirements were supported by Congress. Proponent: DCSPER Report to the House and Senate Committees on Armed Services Defense Manpower Quality Volume II Army Submission May 1985 18 PRODUCT: Army Manpower Cost Modeling System costs of Army Problem/Need/Issue: The Army needs a system for accurately determining all manpower weapon systems, particularly for budgetary purposes early in the acquisition cycle. These costs include enlisted, officer, reserve, and civilian components. ARI Solution: This prototype model provides detailed and comprehensive information on recruiting, separation costs of enlisted personnel. Both a budget model training, compensation, retirement, and and an economic model have been developed. Applications/Results: • Enables planners to make early comprehensive estimates of enlisted manpower costs in the systems acquisition cycle, and permits manpower cost trade-off analysis of alternative weapon systems. • Used to estimate enlisted manpower cost trade-offs between the Bradley Fighting Vehicle and the M113 armored personnel carrier in mechanized infantry battalions. Proponent: ODCSPER DA, DMPB December 85 19 PRODUCT: Recruit Quality and Soldier Performance Model Army Problem/Need/Issue: The Army requires information on performance and performance predictors to set aggregate and MOS-specific recruiting goals for different categories of recruits. The Army must also be prepared to defend the mission for high-quality soldiers to DoD and Congress. ARI Solution: This research investigates measures of soldier productivity, and relates soldier quality factors such as AFQT score and education to these performance measures. By measuring the impact of soldier quality variables on performance, soldiers who have the most potential can be recruited more aggressively, and quality factors can be considered when allocating manpower within the Army. By in~ creasing manpower productivity, readiness will be improved, and training costs will decline. Applications/Results: Used by the ODCSPER to defend the 1986 mission of high-quality soldiers to DoD and to Congress. Proponent: ODCSPER (user -USAREC) December 85 20 PRODUCT: Multivariate Enlistment Model The Army requires an updated methodology to formulate compensation policies Army Problem/Need/Issue: and recruiting strategies relating such factors as unemployment, the relationship of military to civilian pay, educational benefits, bonuses, and the demographic characteristics of the available manpower pool. ARI Solution: This product measures the importance of economic, social, and behavioral factors that impinge on the enlistment decision and provides forecasts of Army quality enlistments. From knowing the effects of pay, educational benefits, bonuses, and recruiting resources on potential recruits by AFQT category under alternative economic scenarios, an optimal (least cost) force can be targeted and recruited, resulting in substantial dollar savings. ARMY GSM 1-3A CONTRACTS Applications/Results: Against Military Civilian Pay Ratio • Used by the ODCSPER to defend the FY 86 budget 1 for military pay and bonuses and allocation of 180001 recruiting resources. :: f\A 84:: • Used by USAREC to forecast enlistments and to 138001 !; '~--\} ,.--"< 180 defend recruiting budget for FY 86. en ' '1 .' 79 t; 12400 ..,, ~ + 78 Proponent: DCSPER (DMPM, HRD) , U.S. Army, USAREC ~ 11000 -1-i u ~ 9600 --·\PAY // 17l ' 76 8200 ,' 7~ "'·-\ : ~--v...-·--' S400 .V ~-.--.. 74 6800 CONTR\TS 73 72 4000 1977 1978 1979 1980 1981 1982 1983 1984 1985 SEA$0NALLY ADJUSTED FISCAL DATA December 85 21 PRODUCT: Multivariate Reenlistment Model Army Problem/Need/Issue: The Army requires a methodology to formulate pay policies and reenlistment strategies to ensure the right number and quality of soldiers reenlisting, relating such factors as selective reenlistment bonuses (SRB), unemployment, and military pay. ARI Solution: This product measures the importance of economic and other factors on the decision of individuals to reenlist, extend, or separate from the Army. From knowing the effects of pay, selective reenlistment bonuses, and other factors on the propensities to reenlist or extend, an efficient SRB program can be developed and managed given budget constraints. This can result in considerable savings because the Army will have a least-cost formula for retaining the number and quality of soldiers it needs. Applications/Results: HIGHER QUALITY SOLDIERS REENLIST AT LOWER RATES •• ~ ~.P • Used to set optimum SRB and pay policies ......: 42 -:// to maximize reenlistment rates by military occupational specialties. ~· 40 //,.- • Used to defend FY 86 budget for amount of i bonuses. ! 38 s ·-·-·-·-·-·-·-·---·/~;(/ U) -_./ Proponent: ODCSPER (DMPB) , U.S. Army ~ 36 ~ ~- 134 .El ~ Legend 32 • CAT! 0 ~~~.~~----30 • CAT lilA 0 CAT 1118 6. ~T~ 28~----------------------.---------------------~ 1983 1984 1985" FISCAL YEAR (•Data thru May 85 only) 22 PRODUCT: Delayed Entry Program (DEP) Management Model Army Problem/Need/Issue: The Delayed Entry Program (DEP) is used by virtually all recruits. Indi viduals sign an enlistment contract and enter the DEP until training begins. Previous work at ARI has shown that time in the DEP affects loss behavior both before and after accession. A system was necessary to take these factors into account and decide how long to place individuals in the DEP. ARI Solution: The DEP Loss-Attrition Tradeoff Model enables the Army to determine efficient DEP enlistments for different types of individuals. Using probabilistic estimates, the model can provide solutions that minimize recruiting and training costs simultaneously. Applications/Results: • The DEP Loss-Attrition Tradeoff Model has been used by the u.s. Army Recruiting Command (USAREC) as a guideline in the formulation of DEP policy. • Results from the model will provide DEP recommendations for the Enlisted Personnel Allocation System (EPAS) currently under development at ARI. Proponent: u.s. Army Recruiting Command TOTAL LQSS COMPOSITION Male I-IIlA H. S. Graduate, MOS 31M .40 .30 Total iii"' a: 0 :z .20 _ --------<:::: OEP Loss -' ::> In-Service Attrition .10 2 3 4 5 6 7 8 9 Months in the OEP December 85 23 PRODUCT: Computerized Adaptive Screening Test (CAST) Army Problem/Need/Issue: The Army faces a continuing and increasing demand to recruit quality applicants. Recruiters need an accurate, efficient screening test to predict the recruit's Armed Forces Qualification Test (AFQT) scores. The screening test must be easy to administer and score, take a of time, and produce results that the recruiter can readily use to tailor the presentation to minimum the prospective recruit. ARI Solution: CAST is a computerized adaptive screening test that is administered to applicants at the recruiting station. The test takes approximately 12 minutes to administer and score and produces an accurate prediction of the applicant's AFQT category. With this information, the recruiter can determine the quality of the prospective enlistee and tailor the recruiting presentation. Based on these enlistment incentives may be discussed. results, applicants may be sent for further testing and Applications/Results: • Recruiters used the test for applicant screening at recruiting stations nation-wide during FY 84. determine which applicants qualify for enlistment and should be sent to a Military • Recruiters can Entrance and Processing Station (MEPS) for additional testing. • Recruiters can reduce recruiting costs. Proponent: u.s. Army Recruiting Command 24 December 84 PRODUCT: Data on Why People Join the Army in socioeconomic factors that Army Problem/Need/Issue: The Army needs annual data to track changes (e.g., advertising and in impact on recruitment and to monitor the impact of Army enlistment programs of enlistment behavior. centives). Additionally, data are needed to develop better predictor models individual enlistment decision-making Previous periodic data collection efforts have not focused on processes. ARI Solution: Information on the reasons for enlistment, characteristics of enlistees in the active now Army, Reserve, and the National Guard, and data on the efficacy of various recruiting strategies is collected at regular intervals. Questionnaires are administered to new recruits. These data are used to determine why people join the Army and to adjust recruitment policies to meet the needs. Applications/Results: • An accurate demographic profile of Army acquisitions can be developed. • Advertising can be targeted to previously undersold markets. was $500,000 • The estimated savings in advertising costs through accurately targeting the market in FY 83. Proponent: Deputy Chief of Staff for Personnel, Department of the Army 25 December 84 PRODUCT: New Armed Services Vocational Aptitude Battery (ASVAB) Composites Army Problem/Need/Issue: Some ASVAB composites were related more to technical school success than to potential on-the-job performance. As a consequence, the ASVAB was a better predictor of technical school success than it was a predictor of job success. ARI Solution: The ASVAB was reevaluated and two composites anchored to the Skills Qualification Test, a measure of job performance. Approximately 100 entry-level MOS were covered including combat arms specialties, such as Infantryman (11B), Cannon Crewman (13B), and Tactical Fire Direction System (TACFIRE) Operations Specialist (13C). The new weighting scheme results in an ASVAB that is a better predictor of both potential on-the-job performance and training school success than its predecessor. Applications/Results: PREDICTING PERFORMANCE • Accepted for FY85 implementation for the selection, classification, and assignment of Army enlisted personnel. • Better match between recruit abilities and MOS performance requirements. • Increased performance worth over $5,000,000 during first year for 54,295 Army accessions in 30 MOS. Proponent: Deputy Chief of Staff for Personnel, Department of the Army December 84 26 PRODUCT: A Profile of Army Families in USAREUR ' Army Problem/Need/Issue: The retention of married Army personnel in overseas assignments is influ enced, in part, by the families' levels of satisfaction with Army life. The Army is considering ways of potentially raising family satisfaction. Before action is taken, however, baseline data are required to ascertain what should and can be done. Baseline data are also necessary to assess the impact of any initiated family-related policy changes. ARI Solution: A family survey was developed and data were collected on the views and perceptions of over 1,000 families with respect to Army life and family experiences associated with overseas assignments. Out of these data, a compendium of information was established that addresses the issues of family wellness, partnership with the Army, and a sense of community identified as important in the white paper; "The Army Family," issued by the Chief of Staff of the Army (August 1983). Applications/Results: • Used by the Department of the Army in 1984 Congressional hearings on family housing, family services, and educational opportunities for dependents; further applications are for policy decision making. • Aided the Community and Family Policy WHAT ONE THING WOULD STOP YOU FROM EXTENDING?. Division, DCSPER, in establishing an Army Family Action Plans Program. PERCENT OF SAMPLE • Potential for reducing permanent change of station costs because of increased satisfaction. NON COMMISSIONED ·OFFICER • Potential for decreasing training costs because of increase in retention. Proponents: Deputy Chief of Staff for Personnel, Department of the Army; Chief of Chaplains, Department of the Army December 84 27 Train the Force PRODUCT: Multipurpose Arcade Combat Simulator (MACS) Army Problem/Need/Issue: MACS was developed as an inexpensive and practical solution to a variety of training problems and issues: the high cost of ammunition, the high cost of currently available fulltask simulators, the lack of live-fire range availability in some areas and for some units, the difficulty of training a variety of tasks with limited performance feedback, and the increasing emphasis being placed on emerging light forces doctrine and associated weaponry. ARI Solution: MACS is a prototype part-task trainer that uses inexpensive, off-the-shelf components. MACS has been designed so that the same general hardware can provide training on different weapon systems by simply moving the sensor (i.e., an optically enhanced light pen) from one weapon to another and changing the software. Currently, MACS software has been developed for the M16A1 rifle, MK19 40mm automatic grenade launcher, M72A2 LAW, and M203 grenade launcher. MACS provides immediate, precise, easy-to-understand performance feedback, varying levels of difficulty, multiple targets and target backgrounds, and it can be used to diagnose a variety of marksmanship errors. Applications/Results: • Performance on MACS for the M16A1 rifle is highly correlated with performance on full-task simulators and moderately correlated with live-fire performance. • In use at Fort Benning and Fort Jackson, MACS has received overwhelming user acceptance. • MACS can be used for diagnostic purposes, as an enhancement in preparatory marksmanship training, for reinforcement training, and as a predictor of live-fire performance. • MACS costs less than $1,000. Proponent: u.s. Army Infantry School December 85 31 PRODUCT: Combat Vehicle lndentification (CVI) Army Problem/Need/Issue: To respond effectively to Army-wide training requirements in the area of target acquisition ~nd analysis, an integrated approach is needed that provides effective and timely training research and development. ARI Solution: Within the context of the Target Acquisition and Analysis Training System (TAATS), the Fort Hood Field Unit has established an ongoing R&D program to address pertinent training requirements and to develop cost-effective training solutions in the areas of target detection, recognition and identification (R&I), and analysis. This year TAATS has focused on the following: (1) providing a mechanism whereby command can emphasize TAATS, (2) formulating guidelines for soldiers in how to use thermal imagery equipment, (3) determining the best training media, (4) establishing the optimal polarity setting to use with thermal equipment, (5) exploring the value of motion for facilitating learning and retention, and (6) developing a a basis for training air-to-ground_ recognition and identification (R&I). Applications/Results: • Developed a videotape entitled "Training for Combat: The CVI Program Series"--an orientation program for use by TRADOC to make students in their various schools aware of R&I-training programs in use Army-wide. • Developed a videotape entitled "A Thermal Sight Adjustment Program" that provides soldiers with a set of guidelines for more effective use of thermal imagery equipment. • Verified that the use of sound filmstrip media for the Basic CVI Training Program is a cost-effective approach for R&I training that overcomes several problems associated with the 35mm slide system. • Image quality of Beseler, video, and 35mm slides appears satisfactory for training; however, the Beseler system has frequent malfunctions and the video system is not currently cost effective (to develop, or use at the unit level). • Determined that when thermal imagery equipment is used, the black hot image polarity setting generally leads to superior identification performance. • verified that use of motion for R&I training generally is not cost effective. • Established that training 15 vehicles over three training sessions is a cost-effective way to train soldier R&I skills. Proponent: Combined Arms Center (CAC) December 85 32 PRODUCT: Training Requirements for NBC and the Future Integrated Battlefield Army Problem/Need/Issue: The Army's inability to train as it expects to fight on future battlefields has been noted in numerous mission area analyses (Close Combat Heavy (CCH); Nuclear, Biological and Chemical (NBC); Close Combat Light (CCL)). This lack of adequate training is due in large measure to the failure to take full advantage of simulation in the development of training for conditions either too costly or too hazardous to reproduce. ARI Solution: The objective is to enhance soldier preparedness through the identification of future battlefield conditions and the methods for training to meet those conditions. Simulation techniques are exploited to provide this training and to evaluate human (operator) performance under the threat conditions of the future battlefield. Applications/Results: • Classified research products specifying tank crew performance capability under conditions that de grade both direct-and indirect-view tank optics were provided to members of the combat modeling and intelligence communities. • Research products for training platoon leaders in Division 86 doctrine and organizations using mission training plans and Dunn-Kempf equipment are available in both manual and computer assisted formats. They have been requested by and provided to the Armor School Command and Staff Department at Fort Knox, the 7th Army Training Command-USAREUR, and the c2 Microcom puter Users Group at Fort Leavenworth. • Information requirements, user requirements, and operator task analyses were developed to better define the battlefield management system (BMS) specifications for the M1 tank product improve ment program. These have been turned over to the USAARMC Director of Combat Developments. Proponent: USAARMC (Armor Center) December 85 33 PRODUCT: Basic Skills Resource Center (BSRC) Army Problem/Need/Issue: The Army needs a mechanism for disseminating information on research in basic skills education to military educators, and for advancing and coordinating research on the use of learning strategies in basic skills education. ARI Solution: To fulfill the information dissemination function, the Military Educator's Resource Network (MERN) was developed by the Basic Skills Resource Center (BSRC), supported by an ARI contract. The MERN consists of a computerized data base on resources in basic skills education, an inquiry response mechanism, and a variety of outreach publications. Tb fulfill the research function, BSRC supported five research studies of learning strategies that produced prototype learning strategytraining in the diverse areas of comprehending technical text, improving basic reading skills, improving self-motivational skills, and learning English as a second language. In addition, this lat ter project produced a guide, "Learning Strategies in English as a Second Language: A Tea~her's Guide," that has been published by the National Clearinghouse for Bilingual Education. This Clearinghouse is funded by the u.s. Department of Education. Applications/Results: • The MERN data base, operating instructions, and procedures were turned over to DANTES (Defense Activity for Non-Traditional Education Support) for defense-wide implementation in FY 86. • Techniques from the prototype learning strategy programs are being incorporated in design of the Army's Job Skills Education Program (JSEP). • The ESL teachers' learning strategy guidebook presents illustrative examples and lesson plans that enable teachers to integrate strategy instruction into ongoing ESL programs. Proponent: Education Directorate, ODCSPER 34 December 85 PRODUCT: Device Effectiveness Forecasting Procedure (DEFT) Army Problem/Need/Issue: The Army needs a formal analytic procedure that can be used at various stages in the design/analytic process to evaluate proposed training device design, proposed design changes, and alternative device designs. This is especially important when field tests cannot be performed due to logistical problems, or when the field tests come too late in the acquisition process to allow change of the device design. ARI Solution: The Device Effectiveness Forecasting Procedure (DEFT) is based on a review of relevant literature and on the development of a conceptual approach that takes into consideration theoretical and practical issues of training device design, development, and evaluation. DEFT is computerized in an interactive menu-driven format for use on the IBM PC or compatible computers. It requires ratings from subject matter experts on device, operational equipment, trainee, and training system characteristics. It can be used at three levels of complexity, varying with the amount of information available to the user. The information input into DEFT is converted into several numerical indexes that are used to estimate device effectiveness. Total effectiveness is derived from two major components, acquisition effectiveness and transfer effectiveness. Applications/Results: • Funds have been allocated for an interservice contract effort, supervised by Naval Training Equipment Center (NTEC), starting in FY 86, to field test DEFT. This effort, known as ASTAR, aims to convert DEFT indexes to scales based on training concepts such as trials to mastery and training time. • NTEC's Training Analysis and Evaluation Group has used DEFT to evaluate the 2F64C helicopter trainer, under four different conditions, to assess the usefulness of DEFT. • The Marine Corps will be evaluating DEFT by using it to evaluate the Landing Vehicle Trainer. Proponent: PM TRADE December 85 35 PRODUCT: Continuous Operations Guidelines Army Problem/Need/Issue: Conflict in Central Europe against forces of the Warsaw Pact has been called the most demanding mission that the u.s. Army could be assigned, but a continuous mission may be required anywhere in the world. In any such conflict, combat operations are expected to continue around the clock con at a high pace. Soldiers will be required to fight without letup for extended period~. Under these ditions, the soldiers' performance will suffer. The procedures for ensuring soldier performance capabil ity and for conserving it in battle need to be well understood. on stress and continuous operations were collated and integrated, ARI Solution: Existing research findings and applied to battlefield scenarios taken from Army field manuals. For continuous operations, a committee Institute of Research developed a of representatives from ARI, Soldier Support Center, and Walter Reed Army The initial output was a three number of simulation models to estimate performance degradation over time. Management Guide, and Technical Documentation. Subsequent work volume set: Human Performance Guidelines, "Soldier Performance in Continuous Operations." produced "Management of Stress in Army Operations" and The latter document is augmented by an administrative manual for command and staff personnel, and one for platoon and squad personnel, containing directions for presenting instruction. Applications/Results: TWo of the above documents have been published as Army Field Manuals: FM 26-2, "Management of Stress in Army Operations," and FM 22-9, "Soldier Performance in Continuous Operations," both December 1983. • FM 22-9 is aimed primarily at Battalion level and deals with methods for sustaining soldiers' performance during continuous operations. It serves several purposes: It presents principles for countering and slowing the erosion of soldier resources. It shows methods of conditioning units for continuous operations and for preserving their fighting capabilities during continuous combat. It presents factors governing rates and degrees of progressive performance degradation in detail. It illustrates differences in expected effectiveness for various types of combat activity over 120 continuous mission hours. • FM 26-2 is designed principally for use by military personnel at Company level and below, and includes sources and degrees of sections on effects of stress in the soldier; stress as a military problem; stress in military life; stress recognition, control and coping techniques; and stress management for leaders. It also includes appendixes on signs of stress in yourself, in others, and in the unit; and techniques for coping with stress in self and others. Proponent: u.s. Army Soldier Support Center 36 December 85 PRODUCT: Design for MOS 19K Basic Noncommissioned Officer Course Secretary of Defense directive to increase emphasis on Army Problem/Need/Issue: In response to a training technology, the Training and Doctrine Command elected to explore training technology applica tions by determining how they might be applied in the MOS 19K Basic Noncommissioned Officer Course (BNCOC). The first requirement was to confirm and update the training needs of the BNCOC student. Since the Army's initial analysis of the 19K duty position was conducted before any soldiers were tanks, it was deemed necessary to examine the results of the initial analysis. actually assigned to M1 In this regard, problem solving, Of particular concern was the identification of new critical tasks. Also at the forefront were new decision making, and interactive tasks were of greatest interest. noncritical tasks that the 19K30 might need to perform if he were to employ the products of new tech nology in BNCOC or in training his crew in the unit. ARI Solution: Task documentation for tasks already in the current Program of Instruction (POI) was re Using information from the ARI viewed and new documentation was generated for tasks not in the POI. data base on 19K BNCOC students, the typical entrant was described and course design implications were drawn. Task training priorities were established by having subject matter experts rate the tasks in The relationship terms of their criti~ality and then having training experts select tasks for training. between the procedural tasks and the nonprocedural tasks was examined in order to incorporate the latter The results from the previously described ac into the context of traditional procedural task training. tivities were used to construct a proposed outline for 19K BNCOC. Applications/Results: • The 19K BNCOC design is being used by the Armor Center as input to a new Program of Instruction being developed and staffed by the Directorate of Training and Doctrine. • The 19K BNCOC course design is being used by Fort Knox Training Technology Field Activity to iden tify, acquire, test, evaluate, and implement train ing technology in 19K BNCOC. Proponent: The Training and Doctrine Command's Training Technology Agency and the Armor Center's Of fice of the Assistant Commandant 37 December 85 PRODUCT: Army Job-Relevant Basic Skills Education Program (BSEP) II Curriculum Army Problem/Need/Issue: The Army has been concerned about deficiencies in the basic skills soldiers need for successful training and job performance. Despite the availability of special remedial programs, studies indicate that the basic skills of many soldiers continue to be inadequate. These short comings have been documented in two GAO reports and in special reports to the Secretary of the Army. ARI Solution: ARI developed and tested a prototype basic skills program that uses soldier-relevant materials to improve language, reading, and math skills. Applications/Results: • Soldiers completing the program significantly increased their scores in reading, language, and math. • FORSCOM and TRADOC began implementing the program in 1983 and are using it as their standard basic skills program in over 50 education centers. • FORSCOM is expanding the program to provide literacy training for NCOs at NCO academies. Proponent: Education Division, ODCSPER COMPUTATION ENGLISH ~I ASA . SECOND LANGUAGE . I December 85 38 PRODUCT: Transitional Performance Aid (TPA) Army Problem/Need/Issue: Field commanders want improved performance from newly graduated Equipment Records and Parts Specialists (MOS 76C), who operate in diverse situations (four duty positions), often without personal guidance, in a highly pressure-laden environment where task performance is specified by numerous technical documents. The uncertainty of assigned duty position and the limited training time available restricts the feasibility of acquiring the degree of unaided proficiency the field commander expects. ARI Solution: The Transitional Performance Aid (TPA) is a job aid that organizes the tasks of the 76C and shows how the tasks are accomplished. The TPA consists of flow charts showing the procedures followed for task completion and the actions for which the 76C is responsible. The flow charts are keyed to a reference matrix. The reference matrix has the specific Army references (by paragraph) for the tasks of each duty position. The TPA allows the user to quickly determine what must be done (from the flow charts) and how to do it (from the reference matrix). The TPA is paper-based and has a computer data base file that facilitates the updating of both flow charts and task referencing. It organizes tasks and responsibilities; is designed as a training aid and job aid for initial duty assignment; allows quick recognition of procedures and identification of doctrinal guidance; creates a job information resource document; and facilitates updating by use of computer data base. Applications/Results: ARI is working with the Quartermaster Center and TRADOC to incorporate the TPA into the 76C course. The final revision is expected to be available soon. Proponent: u.s. Army Quartermaster Center December 85 39 PRODUCT: Bradley Infantry Fighting Vehicle Procedural Guides Army Problem/Need/Issue: Bradley Infantry Fighting Vehicle technical manuals are chronically out dated and unavailable in adequate numbers. Technical manuals are not designed for quick reference, being overly detailed and repetitious. Reports from the field indicate that excessive times are required for preventive maintenance checks and services. There is a need for a ready'reference for infrequently used critical skills and expedient cross-training. ARI Solution: Critical procedures for Bradley Infantry Fighting Vehicle commander, gunner, and driver are presented in flowchart format with decision trees at choice points. Individual and crew tasks and preventive maintenance checks and services are rendered in a factually complete, pocket-sized format for individual soldier ownership and use. The guides are designed for refreshing skills, crosstraining of crew members, and training in emergency situations. Applications/Results: • Compact, easy-to-use format encourages attention to details often overlooked when one-per-vehicle, bulky technical manuals are used. • Format reduces 578-and 488-page technical manuals to an 85-page commander/gunner guide and a 24-page driver guide with no loss of completeness for critical functions. • Flowchart format clarifies critical decision options and reinforces safety by highlighting warnings. • Pocket-sized format facilitates reduction of time to per form critical combat tasks, e.g., erecting swim barrier, correcting gun malfunction. • Simplified format reduces time required to provide proce dural or specification changes in crews in operational units. Proponent: u.s. Army Infantry School December 84 40 PRODUCT: Bradley Infantry Fighting Vehicle (BIFV) Ammunition Storage and Reload Systems -~~----- Army Problem/Need/Issue: Loading 25mm ammunition into the gun "ready boxes" requires emptying cumbersome shipping containers, linking numerous 15-round belts of ammunition, and packing the ready boxes These factors combine to produce unacceptably high using counting rules easily confused under stress. reload times. Space and clearance constraints likewise reduce rapid reload of the ready box of the 7.62mm machine gun. Downtime required to replenish ammunition in one vehicle reduces platoon firepower by 25 percent and increases crew and vehicle vulnerability. Equipment design, rather than inadequate crew training, is the cause of excessive reload times. ARI identified the human factors shortcomings in the original designs, conceptualized ARI Solution: alternative configurations, modified current ready boxes, and built low-cost, prototype ammunition The prototype systems were assessed for crew performance improvement shipping and storage containers. at Fort Benning. Extensive engineering field tests by the Bradley Infantry Fighting Vehicle manufac turer have been completed. Applications/Results: • Reload time for 25mm ammunition is decreased from 13 minutes to 2.4 minutes, a reduction of 82% for crews given minimal practice on the new system. • Reload time for 7.62mm machine gun ammunition is decreased from 6-8 minutes to 1 minute or less, a savings of up to 88%. • On-board stowage of 25mm ready rounds is increased by 34% and for machine gun ready rounds by 25%, both RAILS ------'-.3 -f-REMOVED-\" -'·~~ -, by using existing space more effectively. ®I ·, ~rr ---====== _-, -~ I --- -' '' -(!, L.r/ ---..!!: ~___':",-~I • High user acceptance due to simplified procedures and training. '-' I I' ' (' \';: Dol \ ··, I 1 \ ', I r:i::" I , ; • Both ready boxes passed rigorous Project Manager \ \ li : [' [l E I' ....__"'''C£ I, L±:!. engineering acceptance tests and are prime candi\II lJf1 ,0\1' HINGE ! 1 dates for inclusion in the Bradley Block Modifica :io' I I tion program to upgrade new production vehicles. ~~~ : lo.' I ~ 1[---PEMOVED ·: i I REf,'CVEO-">. '">· :::!:•o·~ • The 25mm ammunition shipping container concept--r~ .:____, 1 one prelinked belt of 25 rounds--has been favorably evaluated by the AMC Ammunition Logistics Command for Bradley and other vehicle applications. Proponent: u.s. Infantry School and Center 41 December 85 PRODUCT: Grenade Launcher Range-Estimating Device for the M203 Army Problem/Need/Issue: Experienced grenadiers make significant errors in estimating the distance to targets up to 350m away. Errors increase sharply with distance, greatly reducing the probability of a first-round hit with the M203 grenade launcher. A range-estimating device is needed that raises the probability of a hit with this weapon. ARI Solution: ARI developed and tested in the field at Fort Benning a 1" x 3" steel template, which provides for matching man-sized images to hole sizes gauged for various distances. No weapon modification was needed. The greatest improvement in range estimation judgments occurred when using the ranging aid for targets placed inside 200m. There was improved effectiveness, however, at all ranges up to 350m. At a probable cost of under one dollar, used leaf sight on the M203 grenade launcher. the device can easily be substituted for the rarely Applications/Results: • Expected error reduction in estimating range by up to 50% over all distances between 50m and 350m. • Substantial savings in lives and ex pense from the reduced incidence of misdirected rounds. Proponent: u.s. Army Infantry School December 84 (update Dec 85) 42 PRODUCT: Training Materials for the Rapid Train-Up, Cross-Training, and Sustainment of M60A3 Armor Crewmen Army Problem/Need/Issue: The Chief of Armor (CG USAARMC) and the Close Combat Heavy Mission Area Analysis identify a need for rapid train-up and sustainment training materials in Armor Forces. During combat, heavy personnel and equipment losses will require replacements that are not necessarily compatible with the originals. Armor crewmen with combat experience may be used to fill more senior positions within a tank crew, while personnel with different MOS may be used as crewmen "fillers." These personnel will need training materials to rapidly train up, sustain proficiency, and cross-train for more difficult crew position assignment. ARI Solution: A Rapid Train-Up Program was developed, which consists of 79 individual M60A3 Armor crewmen task training modules printed on pocket-sized cards. Each task addresses a critical skill required to operate or maintain the M60A3 Armor tank. Tank Study Guides (10) were developed for selfinstruction on issuing fire commands, degraded gunnery procedures, and multiple-range returns. These booklets present the basic knowledges required to perform tasks and realistic pictorial scenarios for applying those knowledges under situational battlefield conditions. Applications/Results: • Ability to reconstitute the Armored Forces wit~ combat-effective crews increases battlefield survivability. • Ability to sustain crew proficiency while cross training personnel in other duty positions in creases unit cohesion. • Better use of training time and fewer operating hours on critical equipment results in reduced training costs. Proponent: Training and Doctrine Command (TRADOC), Unit Training Support Directorate, u.s. Army Training Center December 85 43 PRODUCT:· Unit Rifle-Marksmanship Training Guide Army Problem/Need/Issue: For several years there has been too little emphasis in the Army on quality marksmanship training. Training has lacked necessary feedback during live firing. There have been too few instructors, often improperly qualified for their responsibilities. Training literature is outdated, inadequate, andjor unavailable. ARI Solution: The Unit Rifle-Marksmanship Training Guide is a "one-step" resource document covering all recent improvements in Basic Rifle Marksmanship' (BRM) and Advanced Rifle Marksmanship (ARM). The guide contains information on all programs, exercises, and principles. It describes when and how to use available devices and aids and offers advice and suggestions for developing and conducting effective marksmanship programs. This ARI product is published as a field circular Army-wide as the definitive marksmanship document for use by commanders, trainers, and instructors. Applications/Results: • Improved knowledge base for all commanders, trainers, and instructors. • Improved combat marksmanship performance for all soldiers. • An approximate 30% record-fire hit increase resulting from the installation of the new ARI marksmanship program in all Army Training Centers. Proponent: u.s. Army Infantry School December 84 44 PRODUCT: Skill Retention Model Army Problem/Need/Issue: One of the training manager's primary responsibilities is assuring that Soldiers forgetsoldiers stay trained on tasks they have learned. This requires refresher training. tasks not practiced in the unit on a regular basis. Unfortunately, unit training resources are scarce. for assisting the training manager in scheduling efficiently and organizing re User-oriented methods fresher training are lacking. ARI Solution: The Skill Retention Model is an easy-to-use method that predicts how rapidly tasks will be rated on how be forgotten over intervals of no practice of up to a year. It requires that tasks This rating is based on whether difficult they are to remember. or not the tasks display characteristics known to influence Performance Prediction Nomograph memory, such as length, performance criteria, mental/physical requirements, and degree to which they are job-aided. Each 100 T 100 (120tUUJ task is rated on a maximum of 10 characteristics yielding a I single score. The lower a task's score, the better it will 110 110 be remembered. The ratings have been validated in the field 10 over the past 3 years and have proven accurate. 80 Ill ~ 70Applications/Results: ~ I 10 I so • Paper-and-pencil and computer-based formats were developed l to help training managers from squad level to brigade decide ! 50 ! 50 what refresher training to include and when. I ., ., • The model provides information (rates and likelihood of 'li task forgetting, retention, and the number of soldiers who i 30 can perform a task correctly at any point in time) to aid ~ 30 unit training managers. 20 • The model produces refresher training schedules that use 10 20 10 training resources optimally. I I • ~«~~• sc:o,. Range Proponent: u.s. Army Training Board • 10 11 12 llonthl Sine• l•t Perlonn•nce 45 December 84 PRODUCT: Unit Fire Direction Center Training Program for the 8 1 mm Mortar Army Problem/Need/Issue: Mortar training in the u.s. Army presently suffers from resource restrictions, making it essential to identify the most efficient and effective training procedures possible. One major problem identified in a training effectiveness analysis of institutional and unit mortar training was the need for an exportable Fire Direction Center {FDC) training course for units. The present research product was developed to provide this for the 81mm mortar. ARI Solution: The Unit FDC training program for the 81mm mortar was developed jointly with the Mortar Division of the Weapons, Gunnery, and Maintenance Department of USAIS. All USAIS institutional training programs that include instruction in Skill Level 2 mortar tasks were reviewed, including BNCOC, the Advanced Non-Commissioned Officer Course {ANCOC), and the Infantry Mortar Platoon Course {IMPC). Due to its comprehensiveness and high level of instructional quality, the FDC portion of IMPC and ANCOC was selected as the model for the design of the unit program. It was reorganized to provide a greater degree of standardization from class to class and from instructor to instructor, and the exportable unit training package was developed. Applications/Results: • The Fire Direction Center {FDC) training program for units was designed for an FDC section chief to use in training fire direc tion computers in all aspects of FDC operations. Instruction is situationally focused on the preparation of observed, modified observed, and surveyed firing charts with the M16 plotting board. The use of these firing charts in both basic and advanced mis sions is covered. Detailed lesson outlines, performance exami nations, and supporting training materials are included. • The program incorporates approved institutional training objec tives and standards of the USAIS {as proponent). The institu tional version of the FDC program was implemented within IMPC and ANCOC in January of 1984, and the unit version has been sub mitted to the USAIS for approval. If adopted, with subsequent distribution to Infantry units Army-wide, it should drastically reduce the amount of time needed by unit trainers for lesson planning and preparation, and it should increase the level of standardization in FDC training across units. Proponent: u.s. Army Infantry School December 85 46 Develop Units PRODUCT: Generic Recommendations for Improving Civilian Performance Appraisal Systems Army Problem/Need/Issue: In its search for ways to improve its performance appraisal system, the Army needs to know what features of systems in other Federal agencies have been particularly effective. The current Army civilian performance appraisal system is frequently seen as cumbersome and ineffective. ARI Solution: ARI has developed generic recommendations for designing more effective, easier-to-use federal agency performance appraisal systems based on a literature review, a survey of federal agencies, and interviews with subject matter experts. The recommendations can be used as guiding principles for improving the Army's civilian performance appraisal system and developing performance ap praisal policies, and also provide useful input to other federal agency performance appraisal system designers and to the Office of Personnel Management. Applications/Results: SATISFACTION with the • The report, "Federal Agency Appraisal Survey: 80 PERFORMANCE APPRAISAL SYSTEM • Summary Information Booklet" has been distributed to a number of federal agencies. The recommendations have been briefed to the Director of the Office of Personnel Management, 8 ~ 60 ~ How sollsfactory Is your PA system: a. Ia raters? b. to rotees? c. In your v1ew? ~ the Director of Army Civilian Personnel, and to 0:: a number of Army civilian personnel managers ~ -40 and planners. w ~ Proponent: Civilian Personnel Directorate, DA ~ 20 0 0:: t.~v.,., u~tsotlstodory Z.unaallatoclo,.., lllbord..-tln• ~•satisfactory 5•-.ery 1aliafaclor' Legend E2l raters ~ CSI rotees Q I rzrsr"D cz;r.,y "6i0 v:.3/37 62/38 62/38 Pts ,;,.,.,.r/lndirrrr Rarin 81/19 or bouer 78/22 76/24 77/23 • I ,.;,,.,.~!.'. t:,,.,.,.,.,,,;,n IRRTU'" u...d, to d•te: 763 or lee~ 761 78 77 -1.0 December 85 52 PRODUCT: Advanced Technology Unit Training and Management System (ATUTMS) Army Problem/Need/Issue: current unit training management doctrine requires units to maintain combat readiness by continuously monitoring unit performance and training, identifying the most critical deficiencies of units and soldiers, and then planning, scheduling, conducting and evaluating training on soldier and leader turnover and turbulence experienced a year-round basis. Given the high levels of by most units, keeping track of training needs on paper is virtually impossible. Likewise, preparation of training schedules that make efficient use of constrained resources such as time, firing ranges, maneuver areas, fuel and ammunition is extremely difficult under the best of circumstances, and units to accommodate unexpected taskings and changes in resource availability. must continually adjust schedules ARI Solution: The purpose of the ATUTMS project was to test the feasibility and utility of employing computer technology in unit training management. The ATUTMS prototype consisted of a VAX 11/750 minitape back-up for mass data storage, 13 terminals hard wired to computer, 485 megabyte hard disk with key offices throughout the test battalion, three portable microcomputers, and, of greatest significance, an off-the-shelf relational data base management system (DBMS), plus word processing, graphics, and electronic mail software. Use of the relational DBMS was critical for avoiding the need to build a DBMS, for enabling rapid soldier training, and for providing system flexibility in creating data tables the ATUTMS was the design of a DBMS that enabled and data base inquiries. Another critical feature of direct, on-line access to information in training, personnel, and equipment/resources. Applications/Results: the Combat Development Experimentation • A formal evaluation of the prototype ATUTMS conducted by Center (CDEC) was overall very positive, as were assessments by the host division for the research (the 9ID Motorized), and the research sponsors, the Army Development and Employment Agency (ADEA), the Army Training Board (ATB), and the Army's Director of Training. This system provided the command and staff agencies within the battalion with immediate, accurate, • and fully accessible data for managing their own functional areas and did so with a minimum of formal programming. It also enabled unit personnel to perform highly sought-after word processing functions and gave the battalion a method, via electronic mail, for networking its communications. • A decision has been made by the Commander, 9th Infantry Division (MTZ) and ADEA, and by the DA, Director of Training, DCSOPS, to create a full division-wide prototype system (as a follow-up to the ATUTMS) for eventual exportation to and adoption by the entire Army. Proponents: TRADOC ATSC/ATB; DCSOPS 53 December 85 PRODUCT: Stratified Systems Theory for Executive Leadership Army Problem/Need/Issue: While the technology for force structuring is well understood, the Armydoes not have a formal doctrine for structuring organizations. Such a doctrine would provide the logicand principles for specifying the size of small combat units; the number of command levels within an organization; the proper organization of line, coordinating staff, special staff, and supporting staff;and the incorporation of information systems, control systems, and planning systems within the defined structure. By so doing, it would make possible the consistent fielding of lean and efficient organizations, particularly TDA organizations, thereby increasing the overall combat effectiveness of any given level of Army authorized strength. ARI Solution: Dr. Elliott Jaques of Brunel University, originator of the General Theory of Bureaucracy, was asked to test the fit between that theory and Army orga~izations. Stratified Systems Theory (SST)emerged as a result. It is a theory of requisite organization structure that specifies what tasks are critical at each level of the organization, and how various organizational components should be broughttogether in order to create an organization that will operate both effectively and efficiently. The theory contains a description of individual capabilities required at each of the command levels in re lation to the tasks that must be accomplished, and cial, and coordinating staff in relation to the a logic for the establishment of supporting, spe command levels. The Army's TOE command structure SYSTEMS LEADERSHIP already conforms to this requisite structure, though the size of the small combat units does CULTUAE-VALUES-SYSTEMS not. The technology has now been developed to design TDA units in terms of the same principles. Applications/Results: • SST is now being used as the set of princi ples to guide the design of the future Army Personnel Command, and for trial reorganization of two major Army Staff sections. • Concept approval by DCSOPS has been obtained for a test of SST with small combat units in FY 86. COMPANY PLATOONISOUAOICREW Proponent: ODCSPER December 85 54 PRODUCT: Lessons Learned During 71D(L) Transition to Light Division .Status Army Problem/Need/Issue: To meet the requirement for a substantial but relatively inexpensive, quick reacting force responsive to regional contingencies without tinkering with deterrent assets, the Army has started to transform a number of Infantry Divisions to an especially light, mobil~, and rapidly deployable configuration, beginning with 7ID. This restructuring will create major organizational changes in elements of the Division and how they work together and may create organizational stresses and problems within and between elements. Since the Division's elements are not converting all at once, but instead at a measured pace, transitional problems experienced by the initial elements undergoing change may yield "lessons learned" to make the conversion of following elements more effective and efficient. 7ID has requested ARI to identify and report these "lessons." ARI Solution: ARI, operating as an extension of 7ID(L) ACofS, G-6 (Transition Officer), has, over the period of a year, reviewed files, collected written comments and issues, and interviewed Division staff and operating elements to identify ongoing problems and potential solutions for application to 7ID(L) transition and to develop and document "lessons learned" to aid follow-on divisions in their transitioning to Light Division status. Applications/Results: • Statements of Lessons Learned are organized for utility around: Transition Phase User Echelon (DivjBde/Bn/Co) User function (S1/S3/S4/Installation) • A draft of the report has been delivered to 7ID; it has been used on an informal basis in planning and policy making. • The findings have applicability to follow-on transition of 25ID, 6ID, and 10ID; the report has been shared with these divisions as well. Proponent: 7ID(L) December 85 PRODUCT: Electronic Clipboard Army Problem/Need/Issue: Evaluation of unit and individual soldier performance is required to esti mate combat readiness of units and to identify training needs. The current manual, paper-based system for recording evaluation data in the field, and the subsequent summarizing of results by hand or by human entry to a computer, are time consuming and prone to errors. ARI Solution: The Electronic Clipboard is a hand-held, field-portable, battery-operated computer unit that: (1) receives evaluation checklists selected from a checklist menu from a unit base-station computer; (2) displays the unit or individual soldier name along with checklists stored in the device and selected by the checklist menu; (3) receives and automatically stores a GO or NO/GO rating for each item on the checklist; and (4) uploads scores automatically into the unit's training d~ta base stored in the base-station computer. The Electronic Clipboard uses state-of-the-art technology including a liquid crystal display (LCD) (with backlighting for night use), a software-driven touch screen in place of a fixed keyboard for data entry and computer control, and CMOS chips for processing and data storage (256K bytes RAM). Applications/Results: • • Successful field test at the u.s. Army Armor Center, Fort Knox, Kentucky. Will guide training and collection/ storage of evaluation data on unit and Jt-h individual performance as part of the Integrated Training Management System (ITMS) at Fort Lewis, Washington. SELECTIVE CHECKLIST ~ AFTER EVALUATION. • Immediately applicable for use in LOADING FROM ABTMS DATA BASE IARTEP, ~ AUTOMATED DATA INPUT TO ABTMS Army units with compatible base DRILL.SMI CHECKLIST AIDED UNIT/SOLDIER station computer systems. PERFORMANCE TRAINING DATA BASE EVALUATION AND • Potential for use by all Army units DATA INPUT when the ITMS is fielded Army-wide. Proponent: Training Technology Agency December 85 56 PRODUCT: Program of Instruction (POl} for Individual Ready Reserve (IRR) Aviator Refresher Training Army Problem/Need/Issue: The Department of the Army initiated the IRR Aviator Refresher Training Program to retain and maintain the flying skills of reservists to make them effective mobilization assets of u.s. Army Forces Command (FORSCOM) units. Individual training units, however, are having difficulty meeting the training requirements of reservists who have not flown in several years. In structor pilots are spending too much time on nonflying instruction. There is no uniform program of instruction for the IRR. Much of the instruction is being done on an inefficient, time-consuming, one-to-one basis. ARI Solution: A program of instruction was developed to aid recovery of old procedural and psychomotor skills and knowledge acquisition of current tactics and doctrine. The POI minimizes the instructor pilot's involvement in the academic, nonflying portion of refresher training. The content is oriented toward an annual self-administered academic program coupled with an active-duty aircraft refresher course of approximately 120 hours. Applications/Results: • Provides qualified replacement aviators to FORSCOM units. • Provides for better management of training assets. • The range and amount of retraining can be estimated given certain reserve characteristics: age, years since last piloted, etc. • FORSCOM sent message announcing adoption of the program for the Army's IRR aviators. Proponents: Deputy chtef of Staff for Operations; Army Reserve Personnel Center December 84 (update Dec 85) 57 PRODUCT: Model for Guiding Implementation, Evaluation and Integration Supportof New Training Products in the Army Army Problem/Need/Issue: Training developments that have received positive reviews and are recom mended for dissemination are often ignored in the field. Feedback from would-be users indicates that the new developments are formulated in ways that make them difficult or impossible to implement given available support. The field views new training developments as more complicated and burdensome than the status quo. ARI Solution: The Model for Guiding Implementation, Evaluation, and Integration Support of New Train ing Products in the Army is a set of procedures that explains how to plan effectively for the develop ment and implementation of a new training procedure. The model also deals with how to monitor progress and sustain the training development until it is integrated into routine Army training. The procedures are based, in part, on ARI's experience in assisting in the implementation of the Multiple Integrated Laser Engagement System (MILES) worldwide. During FY 83, the model procedures were used to guide all plans for the field implementation of the new MILES Air-Ground Engagement Simulation/Air Defense (AGES/AD). This airborne adaptation of MILES was produced and implemented in late 1983 at the National Training Center. Applications/Results: • The procedures have achieved greater troop readiness as indicated in training effectiveness analysis. • The Army now has a method for realistic two-sided tactical combat training, providing vital training feedback through its after-action reviews. Proponent: Training and Doctrine Command (TRADOC), Project for Tactical Engagement Simulation December 84 58 Maintain Force Readiness PRODUCT: Computerized Hand-Held Instructional Prototype {CHIP) Army Problem/Need/Issue: There is a need to reduce training delays and the number of failures in Advanced Individual Training and sustain trained skills in operational units. was developed as an inexpensive prepara ARI Solution: A hand-held, computerized, user-friendly "CHIP" and other services. CHIP is lightweight, compact, tion for, and supplement to, training in the Army case layout. CHIP combines the advantages of and portable, with an ergonomically designed keyboard and technology with the familiarity of book instruction. It may be used in both institutional and field reduced classroom training, less time away from duty station, environments. Among the benefits are reduced TDY expenditures, and improved learning and retention of technical terms, facts, and procedures, which can lead to a reduction in maintenance and operation errors and dollar savings in equipment. Applications/Results: • Prototypes will be used to teach technical vocabulary, mathematics, and job-step procedures to sol diers, airmen, and sailors at five sites across the services--including the Army's Air Defense School at Aberdeen Proving Ground. Artillery School at Fort Bliss and the Ordnance • Twenty-five percent increase in knowledge of job related vocabulary and 90% retention after 4 weeks of subsequent training, based on tests with the Hand-Held Tutor. • Cost of $650, including one curriculum, in purchase -~ -~~0 volume of 100 units, with a target cost of $200 in \ quantities of 10,000. ~\ • Usage and completion rates with CHIP are double those of comparable paper-based delivery systems with the same content. Proponent: The Air Defense Artillery School, Ft. Bliss December 85 61