Application Story
Shared Scenario Library as key to Engineering Success of ADAS and AD System: Consistent, Transparent and Interoperable
CARIZON - Shared Scenario Library as key to Engineering Success of ADAS and AD System: Consistent, Transparent and Interoperable
Summary
Background
As newly founded Joint Venture, CARIZON’s key business is to provide highly intelligent ADAS system to Volkswagen Group in China on multiple carlines with rich variance. New business places high demand on a digital framework that facilitates scenario data exchange and interoperation. Further, a more intelligent and efficient solution that can bridge data- and knowledge-driven approaches is needed to accelerate product iteration from both directions. Therefore, we initialized this research project.
Solution
Functional scenarios defined by a product owner help as starting point. Logic scenarios are then derived by system engineers and used to generate Simulation data set. On the other side, testing team applies reverse mechanism to real world sensor data collected from test fleets, from which concrete scenario is extracted and described in standardized format. In the same data base, scenarios from both ends of V-Model are analyzed, aggregated and associated altogether. Based on these scenarios, simulated data and collected real-world data are annotated, sorted, enhanced and clustered in a data lake, used in algorithm training, evaluation, and end-to-end system Validation. Accepted new software release is then deployed back to test infrastructure to close the loop. Iteration by iteration, flowing cross teams, scenarios and data sets are shared and properly used in all development activities to shorten the development lifecycle and enhance the product performance.
INITIAL SITUATION
Complexity of ADAS/AD systems is increasing, and at the same time the pressure to reduce costs is increasing. To deliver products faster than competitors with better quality and cheaper quotes, advanced engineering level is essential. CARIZON has both strength and weakness: data lake infrastructure and Simulation platform are productive, but the data sets are not yet managed in scenario context; scenarios on different layers have been well-documented, however maintained as data islands in different teams and lack of consistency; similarly, we do have acceptance criteria for some modules, nevertheless a retrievable evaluation index system is missing. As a result, huge effort was wasted in communication, especially while switching from one product line to another, to share and/or re-use these data.
CHALLENGE DURING THE PROJECT
Scenario and tags should be described with standardized and machine-understandable language that is extensible, continuously updated along with ADAS/AD technology evolution, commonly accepted in industry. To share scenario library and data set, a user-friendly graphic UI for sorting, filtering, preview, editing and exporting should be provided. Not just function-related evaluation index, but also non-functional index e.g. Run-time performance, CPU, memory utilization should be considered as edge case and corner case candidates. The same functional scenario might derive several logic scenarios with a variety of parameter sets, thus scenario lineage should be described and tracked.
Success Strategy
The solution can be divided into the following steps.
Step 1:
Based on product portfolio, functional scenarios are extracted and described in ASAM OpenDrive, ASAM OpenCRG and ASAM OpenScenario. The Product team uses these scenarios for system requirement specification, based on which the engineering team conducts initial solution architecture design. Sensor simulation is carefully implemented as proof of concept for perception sub-system. Furthermore, sensor technical specification and installation layout are used to analyze edge cases of perception.
Step 2:
Scenarios are reviewed and updated referring to public accident libraries, international and national standards. Legacy projects’ valuable scenarios are also counted. Based on functional scenarios, product line specific system evaluation indexes are added to logic scenarios. The first stable version is then distributed to development and testing teams.
Step 3:
Based on logic scenarios for whole system, module-based evaluation indexes in individual scenario are inferred. Some variables are used as touchstones to explore system functional behavior in certain scenarios, e.g. ego-car speed and acceleration, while others are used to check non-functional performance e.g. a system module's worst case processing time in an intersection with heavy traffic. These are critical inputs for system design, implementation and validation on every layer in V-Model.
Step 4:
Starting evaluation against system component before system integration with edge cases generated from hybrid real and simulated data. Validating the overall system end-to-end performance in real vehicles. Analyzing evaluation reports, adjusting scenarios and evaluation indexes whenever necessary.
Step 5:
Annotating selected test fleet data sets, typically bad cases and bugs with help of ASAM OpenLABEL. These valuable cases are analyzed and described in ASAM OpenScenario XML as concrete scenario, from which logic scenario with corresponding parameter sets and evaluation indexes is distilled and stored in form of ASAM OpenScenario DSL. Such scenarios are more generic and easy to customize due to its Python-wise grammar, thus widely used for cross-system simulation, evaluation and validation, while results could be traced back to referred scenarios on all layers.
We chose ASAM OpenX standards not just because of its well-defined data structure but also the existing ASAM OpenX community. Familiar Simulation platform and toolchain can be enrolled immediately, time and effort are saved. We are looking forward to working closely with other partners and making active contributions to ASAM OpenX standards development.
YANLE ZHANG, Head of System & Architect, CARIZON (Beijing) Technology Company Ltd.
Key Benefits
The project has proven that data-driven and knowledge-driven methodology work perfectly together in engineering world. Driven by knowledge from top side down we go through functional, abstract, logic and concrete scenarios, whereas driven by data, bottom side up, we recognize logic scenarios from concrete one, link to corresponding functional scenario based on well-maintained scenario lineage. Furthermore, the experience of this project helps validate the idea of "Lean Management" against individual system modules. Since evaluation indexes and variable ranges are broken-down to module-level, 6σ index e.g. upper/lower control limit can be applied to lower system level. Now, featured product configuration management is much easier: centrally managed scenarios with evaluation indexes are transparent to engineering teams, changes could take effect without delay or misunderstanding.
OUTLOOK
This is not a single success story of ASAM OpenX standards. As machine-readable language, OpenX standards have digitalized important project data assets into a structured data format, making it possible to involve new automation and artificial intelligence technology in the picture. As a next step, we will record and analyze distribution and utilization statistics of scenarios and data sets, so that data quality, scenario library, development process and workflow, eventually the overall product performance could be continuously improved.
BACK TO OVERVIEW
Follow us on
