Application Story

dSPACE software-in-the-loop (SIL) sensor Simulation is used at Hella to test and validate sensor software in the early stages of the development cycle. In order to compare the computational output of the software with the expected results, high-quality ground truth information is needed from the Simulation.

In addition to the simulated sensor environment data, the dSPACE Simulation models provide time-synchronous ground truth information using the ASAM Open-Simulation-Interface (ASAM OSI) Standard. This information is used by Hella to validate the results of their radar perception software stack.

The ASAM OSI Standard provides highly sophisticated options to distribute ground truth information. By obtaining this information, together with the simulated sensor environment from the dSPACE Simulation, Hella is able to perform an accurate Validation of their complex radar-sensor software.

Best strategy: shifting testing to early development cycle stage

A couple of years ago Hella, was one of the first automotive suppliers to recognize the urgent need for SIL testing as an efficient strategy to shift testing efforts into the early stages of the development cycle. With this strategy in place, problems and errors can be revealed and resolved in an early phase. Developers are not limited to unit tests anymore, but they are enabled to perform highly sophisticated tests during development that are otherwise only possible with expensively recorded real-world data.

Based on a good, long-term business relationship, dSPACE and Hella decided to tackle this problem together in a joint project. dSPACE provides an extensive portfolio of Simulation models suited for hardware-in-the-loop (HIL), as well as SIL Validation use-cases, and focused on the field of ADAS/AD and sensor Simulation. Furthermore, the holistic dSPACE toolchain allows users to create complex scenarios and test cases to validate perception algorithms, as well as complete AD driving functionality in closed-loop setups.

Within the joint project, Hella utilizes the powerful dSPACE Simulation toolchain to validate their sensor software, while dSPACE is leveraging Hella‘s years of experience in the field of radar signal processing to continuously improve their physics-based radar Simulation.

Two mandatory requirements for validation

In order to perform an accurate Validation of a perception software stack, based on a SIL Simulation, two important parts are mandatory. The first part is an accurate Simulation of the input of the perception software (i.e. the signals and the environment that the software is perceiving). This information should be as close as possible to reality, because ultimately the validated sensor software is supposed to work in real world scenarios.

For this part, the dSPACE portfolio offers extensive Simulation models, covering vehicledynamics-, traffic-, and of course, physics-based sensor Simulation. The second part is providing accurate ground truth information that can be used to validate the perception results of the system-under-test. This ground truth information needs to meet certain criteria. It should be a superset of the output of the perception stack, which means all types of signals that are part of the result should be also contained within the ground truth data. This allows a matching of results and ground truth information, and thus, a Validation of the results. Furthermore, it is important that the ground truth information corresponds to the result data (i.e. it is originated from the same point in time).

Solution approach based on standards using ASAM OSI-GT

From the beginning of the project, it became clear that it is desirable to apply a standardized solution for providing ground truth data to the system-under-test. 

ASAM OSI provides a generic interface for the environment perception in virtual scenarios. The contained ground truth interface (ASAM OSI-GT) provides sophisticated information about all objects within the simulated scenario in a global coordinate system, and is therefore, not only suited for Validation of perception algorithms, but also for the Validation of sensor fusion applications.

The ASAM OSI-GT interface was suited for the project for multiple reasons. The dSPACE Simulation environment was already generating ASAM OSI-GT-compliant data, which had been used in other areas of application. By applying the already existing solution to this project, significant time, and therefore cost savings, could be achieved.

The “time-to-action” was significantly reduced as well. By agreeing on using ASAM OSI, both parties were able to directly start with the necessary implementations due to the well documented and standardized ASAM OSI-GT interface without having to work out an interface description from scratch. In addition, ASAM OSI can be used directly with C++ programming language, which is heavily used in Hella’s system-under-test.

In a short time, Hella was able to utilize the ASAM OSI ground truth provided from the dSPACE Simulation environment to validate the output of their radar perception software. The following images illustrate the Validation process within the Hella software, based on an example. The Hella software observes the traffic behind the car, based on multiple radar sensors that are equipped at the back of the car. In the visualization, the dotted boundingboxes indicate ground truth information based on ASAM OSI-GT, which are obtained directly from the dSPACE Simulation, whereas the bounding-boxes with solid lines are showing the perception results of the Hella software stack. This information can then be used to judge the quality of the current status of the software.

A challenge: provision of consistent data

During the project it became obvious, that the selected approach for providing ground truth information was the right choice. ASAM OSI was suitable for all intended Validation tasks. It supports all required data to perform a matching of the ground truth data to the perception results from the Hella software stack. An important challenge during the project was to provide all relevant data in a synchronous manner -- even when the data originated from different sources. Matching of results and ground truth data is only meaningful if they belong to the exact same point in time. Therefore, dSPACE implemented a timestamp-based synchronization mechanism into their data exchange solution to provide time-synchronous ASAM OSI ground truth data in all cases (e.g. even when the Simulation is running faster than real time). This mechanism also emerged from the project and became an integral part of the dSPACE solution.

A summary of benefits derived from the dSPACE and Hella project include:

• Gained flexibility: Updates do not break the interface due to backwards compatibility of ASAM OSI.
• Time savings: Using a standard allows both parties to implement against the interface independently from one another.
• Increased quality: ASAM OSI provides a high-quality interface for ground truth data.
• Cost savings: Using a standard allows the re-use of the solution in other projects and areas.