Development Update

1. Introduction

The AV industry moves at a very high pace, one that often far outstrips the usual speed of standards development. For this reason, the ASAM OpenX projects have been generally been kept to 1 year development pipelines, leading to a more rapid release cadence that is better aligned with industry.

At ASAM we felt that even a time frame of 1 year might not be sufficient to keep up with the pace and so decided that even if the OpenX project work is restricted to ASAM members, we need to keep the industry up to speed. This is the first of multiple quarterly updates on the ASAM OpenSCENARIO V2.0 Standard. With it we want to provide insight into the development direction of the format to give the industry as a whole a chance to align with, provide feedback on or participate in the further activity for ASAM OpenSCENARIO V2.0.

2. Project Overview

The ASAM OpenScenario V2.0 standardization project was launched in June. It is split into 5 groups, each of which consists of multiple work packages (WPs) targeting specific work items, see the proposal for details.

Despite the current situation with work from home and no face to face meetings, all 5 groups are making progress towards delivering a first draft of the Standard in Q3 of 2021. The teams are expanding and leveraging the concepts developed in the concept document.

Although ASAM OpenSCENARIO V2.0 is defining a domain specific language as a significant superset of ASAM OpenSCENARIO V1.x, particular emphasis will be placed on identifying migration rules from V1.x to V2.0.

Disclaimer

The content shared in this update stems from ongoing development work and is thus subject to change without notice!

3. Project Timeline

The original project schedule was defined very aggressively, with the project opting to focus on laying solid foundations for continued development projects and also allowing for realignment with the industry. This is a similar approach to that of the ASAM OpenSCENARIO V1.x development path.

A full project meeting was held at the end of September where all groups presented their status. In general, all groups are in or nearing the norming & performing stages (see here) of their development work.

4. Group Status Updates

4.1. Usage & Pragmatics

This group aims to understand the usage of scenarios and the corresponding workflows that would make use of ASAM OpenSCENARIO V2.0. It will release documented workflows and accompanying guidelines on using the Standard in them as well as guidelines on the different levels of scenario abstraction. 

The Glossary created from the Concept stage has been reviewed and finalised.

The group also reviewed and updated the definition of the use cases from the Concept project and have decided on 42 unique Use Cases across 9 categories.

Use Cases are assigned to team members who are currently modeling the workflows for these. The group is aiming to investigate the possibility of having interactive, combined diagrams together with the service provider. One such example is demonstrated below.

4.2. Language Concepts

This is the most modular of the ASAM OpenSCENARIO groups, tasked with the design of the DSL. Many WPs have been making good progress on various design decisions:

Further foundations for the DSL have been laid, building on top of those developed in the concept project. The first drafts of the basic and compound types were completed. The WP on expressions has been along similar lines, defining physical units and basic operators, now looking at atomic expressions and typecasting.

The work on the surface syntax has not started yet as this is not as critical when some more fundamental semantic & pragmatics topics are still open. This will likely begin early 2021.

The WP defining libraries has started working on namespacing and fallback mechanisms. This WP will also be closely aligned with the OpenLabel project group with respect to scenario meta data.

The Semantics working group are currently focussed on putting together an iterative process to formalise the semantics for ASAM OpenSCENARIO V2.0. They have proposed a layering of the concepts for ASAM OpenSCENARIO V2.0 which increase the power, flexibility and complexity of the language (with the final layer being the full language). The next few meetings will be used to describe the semantics for the basic layers.

4.3. Domain Model Group

The Domain Model group has established a goal to create a deliverable that is easily readable for new and veteran users of ASAM OpenSCENARIO. The domain model will provide a clear view of the scope of the language and a basic representation of its structure. Furthermore, the domain model will be used to ensure that ASAM OpenSCENARIO V2.0 is effectively a superset of the functionalities of ASAM OpenSCENARIO V1.0.

During the first quarter, the group reviewed relevant documentation from other bodies (such as BSI PAS 1883 (operational design domain), ISO 345xx (ISO/TC 22/SC 33/WG 9), ISO 8855 (coordinate systems), DIN SAE SPEC 91381), which will help provide content and clarify the scope of the domain in OpenSCENARIO 2.0.

Sources of Information:

• ASAM OpenSCENARIO V1.0​
• ASAM OSI (Open Simulation Interface​)
• ASAM OpenXOntology​
• CARLA​
• M-SDL​
• BSI PAS 1883​
• ISO 3450x​
• stiEF
• ASAM OpenLABEL​

In the next quarter the group will build the first draft of the domain model and identify areas where ontological definitions can enrich the Standard.

Domain Model content categories

• Vehicles​
• Vehicle Actions​
• Pedestrians​
• Pedestrian Actions​
• Environmental Conditions​
• Coordinate Systems​
• Road Abstractions​
• Traffic Signals & Communication​

Some early examples of domain entities and attributes can be found in the concept paper

4.4. Measurement & Success Criteria

The Measurement and success criteria group made good progress on both fronts and was able to reach consensus through meaningful discussions.

4.4.1. Success Criteria

Much progress was made on both syntax and methodology levels.

There was a large amount of discussion here as well as in other ASAM projects on understanding the interaction between a test and a scenario. ASAM has initiated a Test Specification Study Group to more closely investigate this interaction from a testing perspective (See proposal / Signup for the kickoff on 17.11).

The ASAM OpenSCENARIO V2.0 project group has also defined a task force prior to the aforementioned study group. Their goal is to begin understanding certain definitions that are important for the language development

So far, the group has established some base definitions.
(Note: The information below and any further detail is still subject to group discussion and hence subject to change!)

For ASAM OpenSCENARIO V2.0 as a scenario description format, some aspects of the interaction between these terms need to be clear, particularly the question of what types of success criteria are used where. Currently the group is considering two types of success criteria:

1. Test case success criteria that tell the user whether the ego vehicle and its functions performed as expected or not
   The group identifies three sub-levels of criteria:
        -   Global checks – e.g. make sure that there is no collisions at all times
        -   Reusable scenario checks – e.g. in a specific scenario, allow the TTC to be 3s and nowhere else
        -   Test level checks and project specific refinements – e.g. with project x the TTC of 3s is not enough,
            set it to 4s, with only a warning for a 3s TTC.
    
2. Optional scenario validity criteria, that allow one to check whether the scenario was executed according to expectations
   E.g. - Vehicles changing lane to stay on the right may lead to scenario not generating the "dangerous" or "interesting" situation intended

Further refinements or disabling certain checks can be specified with special syntax in order to enable both reuse and flexibility.

Information

ASAM OpenSCENARIO V2.0 will not prescribe where and how to place success criteria, but it will provide multiple example workflows that demonstrate the various approaches. ASAM OpenSCENARIO 2.0 as a language will provide constructs that allow a user to define these as needed. Clarifying these approaches further, or even providing a recommendation on what should be where, will be part of the scope of the Test Specification Study Group. Interfacing to external test case specifications will however, be included in the language.

4.4.2. Coverage

The definition of coverage from the concept project, especially that of functional coverage, was reviewed and refined. The reuse and extendability aspects of coverage were given particular focus.

Information

The exact syntax for defining coverage is still undefined, as the group is waiting for the syntactical building blocks to be defined by the other language working packages.

Special syntax is proposed to override generic coverage definitions in a non-intrusive way to enable project and ODD customization.

6. Closing Words

The current difficulties due to COVID19 faced by everyone around the globe have of course also presented a hurdle. This project consists of over 100 participants, where many have never met face to face or have never worked in an ASAM project. The fact that this group of members, with a truly worldwide representation, have found a productive working mode, are making progress and are still on track for the planned release, speaks volumes about their engagement not to mention the demand in the industry for a Standard like ASAM OpenSCENARIO V2.0.

I look forward to our continued virtual discussions and very much hope to be able to meet all the contributors in person in the not so distant future!

If there any questions or comments, please feel free to contact benjamin.engel(at)asam.net.