Back to ASAM International Conference 2019

Measurable Safety – A Metric Driven Approach for Safety Assessment And Rating of AVs

Gil Amid  
Chief Regulatory Affairs Officer | VP Operations | Co-Founder
Foretellix Ltd 

According to a study by RAND corporation, it will take 11 billion miles of test driving to prove that an AV has a fatality accident failure rate 20% better than a human driver failure rate. The report also said that it would take up to 500 years of test driving for 100 cars driving 24 hours a day, 365 days a year. Note few companies have this size of a fleet and the resources to operate them continuously. 

Even as we add more testing miles, most experts say that the current safety metrics - quantity of miles and the associated number of vehicle disengagements – are deeply flawed as a measure of safety. In fact, many of these miles may not be exercising the combinations of combinations of driving scenarios critical to prove safety.  The industry needs to shift from “Quantity of Miles” to “Quality of Coverage”.  

Quantity of Miles is defined as: physically or virtually logging miles and the associate number of disengagements and/or failure rates. Unfortunately, miles driven and disengagements are not directly correlated to the core and edge case driving scenarios that must be exercised.  

Quality of Coverage is defined as: successfully exercising the scenarios critical for AV safety and extracting the metrics to prove it. This will provide the ‘measurable safety’ that consumers, developers, regulators and insurance companies require. 

The presentation describes how developers can use coverage driven Verification to ensure that autonomous vehicles behave properly in all combinations of combinations of driving scenarios. This includes the scenario coverage metrics required to make a compelling ‘safety case’ to consumers, suppliers, insurance companies and regulators. The end result of this approach is that the core of safety cases can be developed using meaningful metric derived from the Verification and Validation effort.