Application Story

Many companies engaged in the automotive sector, are already in possession of petabytes of sensor recordings from field operation tests. Since physical test drives are still a proven way to conduct testing and Validation of advanced driving assist and automated driving functions, the amount of sensor data is ever increasing. However, one task to solve is to efficiently extract precious insights from the collected data lakes for repeatable future use.

One approach is to store the sensor recordings from all installed sensors in a timely synchronized manner including their internal raw data. Having this collected enables a one-to-one replay of the recorded test drive by feeding the recordings to a Hardware- or Software-in-the-Loop (HIL or SIL) setup, and “replay” a test drive. However, the data produced by a multitude of test vehicles can easily consume petabytes of data storage, where due to the nature of test drives, not every collected minute is worth storing, since the recorded scene is not resembling an interesting situation. A one-hour test drive, where the test vehicle is equipped with 4 cameras and 4 Lidars, easily sums up in more than 300 GB of sensor data. 

An alternative is to develop methods to derive a deterministic description of specific traffic situations or interesting incidents as a reusable scenario, with multiple purposes in mind. Examples are regression testing in Simulation, test drive analysis by extracting key performance indicators from it, manipulation and variation of recorded tests, and finally enriching your scenario Database. 

To realize this, AAI has developed a two-fold approach: 
Firstly, critical traffic situations are identified and classified into scenario categories (according to so-called Pegasus categories  or maneuver types, like cut-in, hard brakes). This happens in Real-Time during the test drive. Offering this, only relevant time frames are marked for later data processing, which saves the time for engineers to carefully review the data recordings afterward. On top, a comparison between the subjective and objective criticality assessment of the traffic situations can be performed during the ongoing test drive.

As a second step, the data matching the previously marked time frames are post-processed by AAI’s Scenario Cloning pipeline.

As a prerequisite, the sensor data, consisting of a minimum of one front camera feed and GPS/IMU data, will be synchronized and the sensor parameters calibrated. To increase the precision and richness of the scenario, multiple cameras and multiple Lidars can be utilized as well. 
A local Map of the test vehicle’s surroundings is generated, and the vehicle is precisely located. All cues from the surrounding important for scenarios are added to the Map.

The next step is to detect and trace all traffic participants within the sensor range of the test vehicle and localize them on the Map.

For ensuring to generate the highest quality scenarios, we are convinced that a final quality check and the possibility to perform manual corrections are important. Therefore, we integrated our annotation tool for enabling the engineer to perform manual corrections as the last step before outputting the maps and scenarios in ASAM OpenDRIVE® and ASAM OpenSCENARIO® formats.

Importing the generated scenarios into a Simulation environment will create a feedback loop. Data can not only be replayed and analyzed but can also be used for conducting deterministically defined test cases.