Microscopic and Macroscopic Risk Metrics for the Safety Validation of Automated Driving

Abstract

Automated Driving is one of the trends in the automobile industry. Latest developments in technology and prototypes suggest that the introduction of automated driving is near. Despite the advances in the systems themselves, the safety approval is still unsolved. Without further research and improvement in verification and validation methods, a safe introduction of automated driving is not justifiable. Currently, the ECE type approval certifies that the system is safe for road operation and that there is no unacceptable risk involved, based on the as-sumption that a human driver is able to control the vehicle and decides about trajectory in critical scenes. For higher automation, human surveillance is not available any more. Hence, new methods of safety approval and risk assessment need to be installed to substitute the current type approval. \nTo begin with, state of the art safety validation methods for automated driving are analyzed and structured to derive open research questions concerning risk metrics. In this thesis, the focus is on scenario-based testing and field-testing. Field-testing means straightforward testing in real traffic. The risk of the system can be estimated in a statistical approach by the occurrence rate of accidents. Scenario-based testing requires the identification of test cases to shift testing from the road to simulation or proving grounds. However, the target of each safety validation is to estimate the risk of the technology ultimately proofing that the safety exceeds the required safety according to all viewpoints in the whole society. In this thesis, two different terms of risk are used. The average risk of a system, e.g. the occurrence rate of fatal accidents, is called macroscopic risk (MaR). The risk in a single traffic scene is called microscopic risk (MiR). Due to the high distance between two accidents in today’s traffic, MaR cannot be estimated without an extensive amount of data. Thus, it requires an enor-mous mileage to gather enough accident data for significant statistic evaluation. Hence, an important research question is how MiR metrics that evaluate the risk of single scenes with-out accidents can be used to extrapolate MaR. Another use of MiR metrics is the identifica-tion of critical scenes in recorded data or online, during test-drives. These data can be used to derive test cases for the scenario-based testing approach. The three most crucial research questions address the definition of MaR requirements and a top-down approach for defining MiR metrics that are eligible to extrapolated MaR from critical scenes and identify test-cases. They will be further refined in the course of this dissertation. The three questions are: \nWhat are the requirements for MiR metrics and how can their eligibility for the ex-trapolation towards MaR and the identification of critical scenes be falsified? \nAs there are many MiR metrics available as state of the art, an assessment process is estab-lished in this thesis that evaluates both use-cases. The metric shall identify scenarios that are highly demanding for the driver or the automation. At the same time, the metric shall describe the risk in a scene, so extrapolation from MiR towards MaR is possible. To evaluate if a metric is eligible for those purposes requirements are defined. As it is challenging to verify if a metric fulfills all requirements, a falsification strategy is established instead that contains two steps: First, test scenes are defined that must be assessed correctly by a metric. If falsifi-cation by the test scenes is not achieved, it is applied on recorded data of human driven traffic. If the true accident rate corresponds to the extrapolation of risk based on the metric within statistical tolerances, the metric’s eligibility is not falsified. As the last falsification step has a high effort, design guidelines are established that lead to a potent metric if followed in the development process. \nWhich methods and metrics can be used to extrapolate MaR of automated driving from critical scenes in field-testing? \nIn field-testing, the safety of a driving system is derived based on the occurrence rate of certain events, e.g. fatal accidents. If the event under investigation is a critical scene instead of an accident, the occurrence rate increases and less mileage is required for the same statisti-cal significance. However, the occurrence of critical scenes alone has no information on accident risk. If it is assumed that criticality above a certain threshold is prevented by the driver if possible, and that the occurrence of criticality above this value can be extrapolated from the observation in a field test, the occurrence of scenes of higher criticality and even accidents could be extrapolated. First, it is investigated if state of the art metrics, data collec-tions and extrapolation methods fulfill the derived requirements. As a result, extreme value theory is selected as statistical tool for extrapolation of risk, assuming that highly critical events are extreme events that cannot be approximated by a fit of all occurring criticality values. Finally, a metric that has proven itself so far, is presented. The metric uses model predictive optimization to find the trajectory with the minimum driving requirements in a given scene. The metric fulfills all defined test cases and compares driving requirements with the estimated human driving skill to describe criticality. Following the established design guidelines, a sensitivity analysis is conducted on uncertain parameters to research the influ-ence in parameter choice on the extrapolation result. For this purpose, the highD-dataset that was recorded from drone footage is analyzed. \nWhat is the acceptable MaR for automated driving? \nThe common expectation is that the introduction of automated driving will reduce the num-ber of accidents at least long-term and per mileage. At the same time, it is obvious that the introduction will induce new risks to the live of modern society, as almost every new tech-nology does. Probably, early adopters will willingly accept risks or uncertainty about risk due to the new experience and the personal benefit. In contrast, passers-by that feel no personal benefit likely have higher requirements. In this thesis, acceptable risks are derived from accident statistics, risk acceptance studies and comparison with other technologies. Based on the requirements, introduction strategies under uncertainty are discussed with the assumption that user are likely to accept the hypothesis that vehicles are safe and fulfill their individual requirements, while passers-by and the society are more likely to reject the hypothesis.