key: cord-0070479-hedt1xcz authors: Boscher, Thomas; Günther, Adrian; Scheck, Korbinian title: Evaluation Method for Manual and Automated Parking Maneuvers date: 2021-11-26 journal: ATZ Worldw DOI: 10.1007/s38311-021-0736-0 sha: a78a3c7a0714185d05c474e547e25e36eca3be02 doc_id: 70479 cord_uid: hedt1xcz nan vehicles. Reasons for this may be the steadily growing population and the accompanying increase in the number of vehicles, especially in cities, and the associated decreasing amount of available parking spaces and possible minimization of parking spaces. As a result, there is a growing desire to make parking more efficient and safer. Although parking is an essential task of daily driving, it is currently rarely the scientific focus of advanced driver assistance systems (ADAS) development. Yet parking has enormous potential to become one of the first SAE Level 3 functions. Preliminary studies as well as analyses by Kempten University of Applied Sciences and MdynamiX show that current PMA versions still have considerable potential for improvement in their functional characteristics and are not satisfactory in terms of customer acceptance. Only if a function can perform its driving task at least as well or better than the driver himself will it generate enthusiasm and thus establish itself on the market. For the performance analysis of PMA, an objective analysis of the basic driving functions is necessary in addition to a subjective evaluation. Only by analyzing with objective parameters of the parking maneuver can the PMA be specifically optimized, contribute to a better subjective experience, and increase customer acceptance. In the development of ADAS, the establishment of brand-specific driving characteristics on the part of automobile manufacturers is also becoming increasingly important. Even for parking systems, different car manufacturers take different approaches to the driving task. With the help of a complete objectification of a parking maneuver, the basis for the validation and definition of manufacturer-specific goals can be created and a brand-independent comparison can be performed. There are few objective evaluations of manual parking maneuvers and PMAs in the literature, especially focusing on detailed objective analysis and direct comparison between systems, humans, or each other. Furthermore, there is never a detailed discussion of the overall steering strategy or the individual phases of a parking maneuver, whether manual or automated. Examples in the literature here are [3] , [4] , or [5] . In a research project of the University of Applied Sciences Kempten and University of Applied Sciences Munich, an objective evaluation method for parking maneuvers was developed [6] . The objective evaluation method of Boscher et al. is based on so-called Performance Indicators (PIs) and Key Performance Indicators (KPIs), which capture the objective overall quality/performance of a parking maneuver on an expert-oriented level in as many facets as possible [6] . The evaluation study focuses primarily on the analysis of longitudinal and lateral driving characteristics with an emphasis on driving comfort and performance. The PIs and KPIs are evaluation indicators that are as detailed as necessary but as superficial as possible. The aim of this new evaluation method was to analyze the parking maneuver in its different phases, from the selection of the parking space to the exit of the parking space, forming the essential interface for the transformation of the subjective user requirements to the development side of PMA. The basis of general parking maneuver evaluation is formed by Subjective Assessment Indicators (SAI), which cover all necessary aspects of a parking maneuver. A detailed introduction to the development of SAIs can be found in [7] . For an evaluation in all relevant areas of a parking maneuver, the evaluation criteria must thereby include not only the vehicle dynamic properties but also the static vehicle position in relation to the environment in the parking space. On the one hand, the SAIs include all necessary customer aspects of a parking maneuver; on the other hand, the SAIs are kept as general as possible and represent only the most important subjective aspects for the performance evaluation of a parking maneuver. The objective analysis of a parking maneuver focusing on the longitudinal as well as lateral driving strategy, is based on the following SAIs from the evaluation scheme of [7] : -steering wheel movement -parking comfort -dynamic performance. The SAI steering wheel movement considers comfort aspects related to the steering wheel. Parking comfort, on the other hand, focuses only on comfort aspects relating to the movements of the vehicle body. In principle, the steering wheel movement can be considered as part of the parking comfort. However, in [7] it is shown that there is no relevant correlation between these SAIs and a separate evaluation is recommended. In summary, the SAIs steering wheel movement and parking comfort can be assigned to the category of driving comfort. The separate consideration of steering wheel movement and body-related driving comfort is particularly well suited as a basis for the development of the PIs of the objective evaluation, since an explicit analysis of the lateral as well as longitudinal driving strategy should be made possible here. Whether the parking maneuver was ultimately performed too fast or too slow is determined by evaluating the SAI dynamic performance. The method for the objective evaluation of parking maneuvers is based on the characteristics of these SAIs [6] . Accordingly, for the objective evaluation, suitable objective evaluation criteria were formed for each of these individual evaluation indicators, which reflect the performance of the respective SAIs in an objective and detailed form. The focus here was on the essential characteristics of the individual SAIs according to [8] , Parking maneuvers in themselves can take many different forms and variations. The basic differences include the degree of automation and the parking geometry. In order to fully compare both manual and partially or fully automated maneuvers in longitudinal or lateral parking spaces, a uniform basis for evaluation is required. For this purpose, a phase model approach serves as a basis, which divides the parking maneuvers into independent sections. FIGURE 2 shows a section of the subdivision into five concise phases that build on each other in terms of time. In order to be able to examine the phenomena in detail and assign them chronologically in the further course of the study, the defined phases are divided into individual sections. These will be labeled with unique designations and individual start and end times will also be defined. The specific parking maneuver in phase 3 represents the primary focus for the development of objective performance indicators. Because of this generic structure of the phase model, any parking maneuver can be analyzed and evaluated. Only some tasks can be neglected for manually performed parking maneuvers, such as the transfer of tasks to the system or from the system to the driver. No specific requirements are given for the way the parking maneuver is performed. Based on the results of the subjective data evaluation in [8] , the PIs and KPIs are determined using correlation analysis from descriptive and intuitive evaluation parameters. For this purpose, individual evaluation formulas were created and adjusted to the most suitable coefficients possible, which will not be discussed in detail here. The detailed formula descriptions and definitions can be found in [6] . The most important KPIs are as follows: -steering wheel angle -steering wheel speed -steering wheel movement consistency -stationary steering -speed consistency -jerk -total time -number of parking moves Parking maneuver study. The objective evaluation method was conducted in a comprehensive driving study in a private parking lot at the Kempten University of Applied Sciences. A total of 21 participants, TABLE 1, took part in the driving tests under special conditions of the Covid-19 pandemic. The participants in this study were primarily semi-experts, that means, engineers with testing experience but not experts who test PMAs on a daily basis, and non-experts. Here, two different vehicles (equipped with extensive measurement equipment) were tested in longitudinal parking situations. Participants performed each parking maneuver both manually and fully automatically. The results of the study are used to analyze and compare the driving strategy of fully automated parking maneuver assistants with the driving strategy of humans themselves, TABLE 1. Each maneuver was performed by the participants three times in a row. Each driving maneuver was recorded from the beginning of a parking maneuver to the final stop in the parking space. The size of the parking spaces was chosen according to the minimum detection gap of the fully automatic parking maneuver assistants. Baumeister's market analysis showed that partially as well as fully automated PMAs are among the least popular driver assistance systems [2] . Subjec- tive analysis of parking maneuver ratings by Scheck and Pfeffer also suggests that the performance of PMAs is not sufficiently superior to human performance [7] . Therefore, for the objective comparison between humans and machines and the associated hypothesis testing, it is necessary to assume that the system performance of PMAs is worse than the performance of manually performed maneuvers. In the study [6] , the following hypotheses were therefore formulated in order to enable conclusions to be drawn regarding the concrete difference in performance between humans and machines, pri-marily in the area of driving comfort (focus on stationary steering) and dynamic performance. Based on the study results and the method developed, the following hypotheses were formulated and tested: -Automated parking maneuvers need more parking moves. -Automated parking maneuvers need more time. -Automated parking maneuvers steer more while stationary. Due to differences in the driving skills and experience of the drivers, the number of manually performed parking operations is broadly distributed, which means that there is no normal distribution of the number of moves. However, FIGURE 3 shows that the automated parking operations also do not have a Gaussian distribution. A Mann-Whitney test shows no significant difference in the number of parking moves required for vehicle A or B. A descriptive evaluation shows that manually performed parking operations require on average about one move more. Since the required parking time shows a mean positive correlation with the number of required parking moves, the distribution can also be considered as not normally distributed, FIGURE 4. However, the difference in parking time between manually and automatically performed maneuvers is significantly different for vehicle A. Parking maneuver assistants require less time for the parking process. The KPI steering while standstill is also not normally distributed in manual and automated parking, FIGURE 5. The evaluation shows that there are significant differences between manual and automated parking with regard to the steering wheel angle during a parking maneuver. It also shows that the steering angle requirement of the two assistants differs significantly. The subjective studies in [7] and [8] served as the basis for the objective analysis of current PMAs in [6] . The method is based on the generically developed phase model for parking operations. The measurement data labeled according to the phase model can be processed and automatically evaluated by the self-developed evaluation software. The soft- ware calculates the KPIs and PIs of the phase model according to self-developed evaluation rules. In this article, only some of the KPIs that are important for customers have been described and the results of the study have been examined in more detail. Significant differences have emerged between manual and fully automated parking processes. The enormous amount of parking steering during the parking process is a clear characteristic of a fully automated parking assistant. The ability to objectively compare manual and partially or fully automated PMAs will support the future development and calibration of new parking assistants. We don't know whether green hydrogen will be the new gold, but we do know that we have a lot to offer the hydrogen industry. We have a great deal of knowledge and expertise in this area, including the ability to think systemically for the automotive and industrial markets. This makes us the perfect partner for our customers -now and in the future. With us, the future is green. As a technology partner, we're constantly challenging ourselves -especially when it comes to green hydrogen. We pioneer motion we-pioneer-motion.com Kundenorientierter Benchmark zum Parkmanöverassistent -Vergleich zwischen Kundenanforderungen und nutzererlebnis mit Hilfe der Quality-Function-Deployment Methode Reinforcement Learning-Based Motion Planning for Automatic Parking System Integrated model of assisted parking system and performance evaluation with entropy weight extended analytic hierarchy process and two-tuple linguistic information Human Based Rating Approach for Automated Valet Parking Function Evaluation Development of an objective evaluation method for manual and automated parking maneuvers. chassis.tech plus Parking -a Subjective Evaluation Method for Manual and Automated Parking Maneuvers Detailed Analysis and Characterization of Subjective Assessment Indicators of Manual and Automated Parking Maneuvers