key: cord-0065268-j6e2d9mr authors: Kremer, Markus; Hamidi, Khalil; Christiaens, Sébastien; Hülshorst, Thomas title: Evaluation of SAE Level 3 Driving Functions on European Roads date: 2021-07-01 journal: ATZ Worldw DOI: 10.1007/s38311-021-0671-0 sha: 57404694c391b8234f85b13696237fee167c0a3b doc_id: 65268 cord_uid: j6e2d9mr nan The key in testing is to ensure that the functionality of the systems used is exposed to varying conditions, and performance is consistent, reliable and predictable. That is the reason why the L3Pilot project is aiming at exposing these systems to different users, mixed traffic environments, including conventional vehicles and vulnerable road users, and ultimately at creating a standardized Europe-wide piloting environment for automated driving. The L3Pilot consortium brings together stakeholders from the whole value chain, Technologies for automated driving must be safe and efficient if they are to reach market maturity. With the participation of FEV, Europe-wide road tests are being conducted and evaluated on a large-scale basis in the L3Pilot EU research project. The FEV test vehicle, its sensors and first experience from collecting data with this vehicle on open roads are presented in this article. www.springerprofessional.com/automotive including OEMs, suppliers, academic and research institutes, infrastructure operators, governmental agencies, the insurance sector and user groups. More than 1000 users will test approximately 100 vehicles across Europe with bases in ten European countries, FIGURE 1. The project started in 2017, will span 48 months and include 18 months of road tests. For the piloting phase, FEV is using one of its automated driving vehicles usually used as a technology and development platform. This vehicle is equipped with the sensor set shown in FIGURE 2 and with appropriate controllers and data logging systems. Thanks to a specific interface to the powertrain, braking and steering systems, engineers can develop functions that assist or even substitute the driver in some specific driving tasks. Next to the sensors used for the environment perception around the vehicle, the vehicle is also equipped with cameras used for monitoring the driver's activities and reactions when they are not actively driving or when they are prompted by the system to take over the driving tasks. These cameras are looking at the driver's face, hands and feet and are logged simultaneously, in order to enable the analysis of the interaction between the driver and the automated driving systems in the different driving conditions. The automated driving applications Parking Chauffeur, Traffic Jam Chauffeur (TJC), Highway Chauffeur, and Urban Chauffeur are analyzed within the scope of the project. The FEV vehicle is equipped with a TJC system developed in-house which can take over control of driving in © FEV traffic jams up to 60 km/h. When the vehicle is driving on a qualified road and the system detects slow-driving vehicles in front, indicating a traffic jam, it will signal the driver that the TJC can be activated. If the driver activates the system, it will take control of the vehicle, follow the current lane and keep a safe distance to the leading vehicle without any input from the driver, until the driver takes over manually again. When the system detects the end of the traffic jam or when the vehicle approaches the end of the highway, it will prompt the driver to take over the control through acoustic and visual signals. The driving tests on open the road are planned to last for at least 200 h in automatic driving mode. During this time, between 10 and 15 drivers will test the the traffic jam chauffeur on German highways around Aachen, Cologne and Düsseldorf, FIGURE 3, and collect data for the following analysis phase of the project. The evaluation process involves the analysis of both subjective data from study participants and members of the public data, for example via questionnaires, and objective data from the pilot vehicles, such as recordings of vehicle CAN bus data. This data will be aggregated for the whole L3Pilot fleet. Based on the objective and subjective data from the pilots, the automated driving functions will be evaluated with respect to their technical and user-related performance. Based on the findings from technical and user and acceptance evaluation, the impact of automated driving functions, in terms of their technical and user-related effects, will be scaled up and evaluated. For the data collection, two data loggers are combined, FIGURE 4. One, hereafter called the ICU, is connected with cloud servers through the mobile network and streams specifically selected data continuously from the vehicle. Another logger is collecting all raw and object data from sensors like cameras, radars, lidar and GPS as well as actuators like the brake and steering control system and stores this data on a hard disk inside the car. The analysis of the data begins as soon as it is delivered through mobile network to the cloud via the ICU. As soon as the data is available on the server, the data is synchronized and converted to the Common Data Format (CDF) [5] defined in the L3 Pilot consortium, based on the open-source Hierarchical Data Format (HDF5). Once the data is prepared, it is transmitted to the selected analysis partner from the consortium who runs it through the comprehensive toolchain framework developed by the partners, based upon tools and experiences gathered in previous projects such as euroFOT [6] and AdaptIVe [7] . Among other evaluation steps, scripts are run which automatically detect specific scenarios defined in the project. When such a scenario is ATZ worldwide 07-08|2021 detected, the corresponding data inside the database is automatically annotated. The aim of this scenario-based assessment is to analyze the behavior of the automated driving system from a technical as well as from a traffic point of view. For that, both Derived Measures (DMs) and Performance Indicators (PIs) are derived from the data and classified by the analysis partner according to the detected scenario [8] . For example, the DMs and PIs are defined to analyze how well the automated driving system can keep a safe distance to the leading vehicle or how softly the vehicle approaches a standing vehicle and then takes off again. The number of take-over requests and the frequency of such requests is also analyzed in detail. To simplify the analysis of these PIs, a web-based dashboard was developed which represents the evaluation of the analyses of all PIs, FIGURE 5. All the data collected during the project will then be analyzed by the respective analysis partner and will be aggregated so that more general findings for the overall piloting fleet can be obtained. Comprehensive testing of automated driving systems under real traffic conditions is an important step in the design and validation of these systems. It provides very valuable data and information for designing and evaluating the technical aspects of these systems. The L3Pilot project will also provide additional information about the user acceptance, driving and travel behavior, as well as impact on traffic and society. The project is paving the way for large-scale field tests of series cars on public roads and provides a solid framework for the necessary harmonization of these activities among the different OEMs and other relevant actors of the automotive industry. Due to the impact of the Covid 19 pandemic, FEV could only gather a small part of the planned amount of test data until the delivery of this article. The amount of collected data was not yet enough to draw any final conclusion. The further progress of the tests and the project will be made available, among others, on the project's website [1] . The future of the internal combustion engine and alternative powertrains L3Pilot Project SAE: Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems. Standard no. J3016 Efficient data management by means of networked Loggers deliverable d4.1 -description and Taxonomy of Automated driving Functions L3Pilot -driving Automation The L3Pilot Common data Format -Enabling Efficient Automated driving data Analysis. 26 th international Technical Conference on the Enhanced Safety of Vehicles (ESV) AdaptiVe Project euroFoT Project The L3Pilot data management Toolchain for a Level 3 Vehicle Automation Pilot The L3Pilot project has received funding from the