B.3 Workflows for auditors/regulators
If you are an auditor or regulator you might be interested in the following workflows.
B.3.1 Workflow list
-
Section B.3.2
As an auditor/regulator, I can understand how AV/ADAS developers are testing their products. -
Section B.3.3
As a safety consultant, I can recommend specific scenarios and related conditions (parameters) to my clients to test their products. -
Section B.3.4
As a government agency, I can understand what parts of the Operational Domain are verified by an AV/ADAS developer through each scenario. -
Section B.3.5
As a regulator/technical service, I can specify, in an unambiguous way, the test scenarios required to demonstrate compliance with a particular regulation. -
Section B.3.6
As a test engineer/auditor/regulator, I can easily trace test cases, scenarios and results back to the requirements.
B.3.3 Recommending scenarios and parameters
B.3.4 Understanding verification coverage
B.3.5 Specifying regulation scenarios
B.3.5.1 Workflow short description
As a regulator/technical service, I can specify, in an unambiguous way, the test scenarios required to demonstrate compliance with a particular regulation.
B.3.5.2 Workflow detailed description
An increasing number of regulation organizations need to provide scenarios for automated or assisted driving functions.
At some point those scenarios must be executed by a piece of software, but often, they are initially written in a human-readable, but not machine-readable way.
As a consequence a regulator or someone who needs to perform the test must interpret and formalize the scenario description first, so that a machine can execute it. This means it must be possible to express the scenarios in a machine-readable, standardized and unambiguous manner.
This use case describes what is needed to turn a formal, but not machine-readable regulation scenario into something that can be executed by software in simulation or on a test track.
B.3.5.4 Steps for specifying regulation scenarios
The following steps are needed:
-
Analyze the setup and content of the regulation.
-
Define what the product must be able to do.
-
Turn the inputs from the first two steps into a scenario definition.
-
Define the parameters of the scenario so that there are no ambiguities left.
-
Execute the scenario.
-
Check the result.
If the test was passed the system is compliant with the regulation.
This use case was added to ensure that OpenSCENARIO 2.0 can be used as a basis for defining executable scenarios based on regulation requirements.
B.3.6 Tracing back requirements
B.3.6.1 Workflow short description
As a test engineer/auditor/regulator, I can easily trace test cases, scenarios and results back to the requirements.
B.3.6.2 Workflow detailed description
During a test campaign, OpenSCENARIO is just the tool used to describe scenarios and execute them in a simulator. At the same time, each one of these scenarios are linked to test cases and requirements.
All in all this use case describes how an OpenSCENARIO user is able to get the results of a test case execution within a simulation and trace it back to the requirements.
B.3.6.4 Steps for tracing back requirements
The person taking these steps is a test engineer that has to link results and requirements after simulator execution. Requirements, results and a scenario database already exist.
-
Get the requirement and the results database.
-
Get the scenario database.
In case it exists, also get the test case database. -
After getting all the databases there shall be a vendor tool facilitating the task of linking results to test cases, and to requirements.
-
The test engineer shall ensure that each one of the test cases references a requirement.
As a result, a requirement for OpenSCENARIO 2.0 is that it shall include a placeholder for requirements that can be later on easily searched in a database.
As a result all the requirements, test cases, scenarios and results are linked.
When the results are analyzed, tracing back the original requirements and the originating test cases can be done at any stage of the validation procedure.