Mykel J. Kochenderfer

Airspace Encounter Models for Estimating Collision Risk

Airspace encounter models, providing a statistical representation of geometries and aircraft behavior during a close encounter, are required to estimate the safety and robustness of collision avoidance systems. Prior encounter models, developed to certify the Traffic Alert and Collision Avoidance System, have been limited in their ability to capture important characteristics of encounters as revealed by recorded surveillance data, do not capture the current mix of aircraft types or noncooperative aircraft, and do not represent more recent airspace procedures. This paper describes a methodology for encounter model construction based on a Bayesian statistical framework connected to an extensive set of national radar data. In addition, this paper provides examples of using several such high-fidelity models to evaluate the safety of collision avoidance systems for manned and unmanned aircraft.

Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks

Deep neural networks have emerged as a widely used and effective means for tackling complex, real-world problems. However, a major obstacle in applying them to safety-critical systems is the great difficulty in providing formal guarantees about their behavior. We present a novel, scalable, and efficient technique for verifying properties of deep neural networks (or providing counter-examples). The technique is based on the simplex method, extended to handle the non-convex Rectified Linear Unit (ReLU) activation function, which is a crucial ingredient in many modern neural networks. The verification procedure tackles neural networks as a whole, without making any simplifying assumptions. We evaluated our technique on a prototype deep neural network implementation of the next-generation airborne collision avoidance system for unmanned aircraft (ACAS Xu). Results show that our technique can successfully prove properties of networks that are an order of magnitude larger than the largest networks verified using existing methods.

Collision Avoidance for Unmanned Aircraft using Markov Decision Processes

we investigate the automatic generation of collision avoidance algorithms given models of aircraft dynamics, sensor performance, and intruder behavior. By formulating the problem of collision avoidance as a Markov Decision Process (MDP) for sensors that provide precise localization of the intruder aircraft, or a Partially Observable Markov Decision Process (POMDP) for sensors that have positional uncertainty or limited eld-of-view constraints, generic MDP/POMDP solvers can be used to generate avoidance strategies that optimize a cost function that balances ight-plan deviation with collision. Experimental results demonstrate the suitability of such an approach using four dierent sensor modalities and a parametric aircraft performance model.

Unmanned Aircraft Collision Avoidance using Continuous-State POMDPs.

An effective collision avoidance system for unmanned aircraft will enable them to fly in civil airspace and greatly expand their applications. One promising approach is to model aircraft collision avoidance as a partially observable Markov decision process (POMDP) and automatically generate the threat resolution logic for the collision avoidance system by solving the POMDP model. However, existing discrete-state POMDP algorithms cannot cope with the high-dimensional state space in collision avoidance POMDPs. Using a recently developed algorithm called Monte Carlo Value Iteration (MCVI), we constructed several continuous-state POMDP models and solved them directly, without discretizing the state space. Simulation results show that our 3-D continuous-state models reduce the collision risk by up to 70 times, compared with earlier 2-D discrete-state POMDP models. The success demonstrates both the benefits of continuous-state POMDP models for collision avoidance systems and the latest algorithmic progress in solving these complex models.

Decentralized control of partially observable Markov decision processes

The focus of this paper is on solving multi-robot planning problems in continuous spaces with partial observability. Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) are general models for multi-robot coordination problems, but representing and solving Dec-POMDPs is often intractable for large problems. To allow for a high-level representation that is natural for multi-robot problems and scalable to large discrete and continuous problems, this paper extends the Dec-POMDP model to the Decentralized Partially Observable Semi-Markov Decision Process (Dec-POSMDP). The Dec-POSMDP formulation allows asynchronous decision-making by the robots, which is crucial in multi-robot domains. We also present an algorithm for solving this Dec-POSMDP which is much more scalable than previous methods since it can incorporate closed-loop belief space macro-actions in planning. These macro-actions are automatically constructed to produce robust solutions. The proposed methods performance is evaluated on a complex multi-robot package delivery problem under uncertainty, showing that our approach can naturally represent multi-robot problems and provide high-quality solutions for large-scale problems.

Accounting for State Uncertainty in Collision Avoidance

An important consideration in the development of aircraft collision avoidance systems is how to account for state uncertainty due to sensor limitations and noise. However, many collision avoidance systems simply use point estimates of the state instead of leveraging the full posterior state distribution. Recently, there has been work on applying decision-theoretic methods to collision avoidance, but the importance of accommodating state uncertainty has not yet been well studied. This paper presents a computationally efficient framework for accounting for state uncertainty based on dynamic programming. Examination of characteristic encounters and Monte Carlo simulations demonstrates that properly handling state uncertainty rather than simply using point estimates can significantly enhance safety and improve robustness to sensor error.

Aircraft Collision Avoidance Using Monte Carlo Real-Time Belief Space Search

The aircraft collision avoidance problem can be formulated using a decision-theoretic planning framework where the optimal behavior requires balancing the competing objectives of avoiding collision and adhering to a flight plan. Due to noise in the sensor measurements and the stochasticity of intruder state trajectories, a natural representation of the problem is as a partially-observable Markov decision process (POMDP), where the underlying state of the system is Markovian and the observations depend probabilistically on the state. Many algorithms for finding approximate solutions to POMDPs exist in the literature, but they typically require discretization of the state and observation spaces. This paper investigates the introduction of a sample-based representation of state uncertainty to an existing algorithm called Real-Time Belief Space Search (RTBSS), which leverages branch-and-bound pruning to make searching the belief space for the optimal action more efficient. The resulting algorithm, called Monte Carlo Real-Time Belief Space Search (MC-RTBSS), is demonstrated on encounter scenarios in simulation using a beacon-based surveillance system and a probabilistic intruder model derived from recorded radar data.

A decision-theoretic approach to developing robust collision avoidance logic

All large transport aircraft are required to be equipped with a collision avoidance system that instructs pilots how to maneuver to avoid collision with other aircraft. The uncertainty in the behavior of the intruding aircraft makes developing a robust collision avoidance logic challenging. This paper presents an automated approach for optimizing collision avoidance logic based on probabilistic models of aircraft behavior and a performance metric that balances the competing objectives of maximizing safety and minimizing alert rate. The approach involves framing the problem of collision avoidance as a Markov decision process that is solved using dynamic programming. Although this paper focuses on airborne collision avoidance for manned aircraft, the methods may be applied to collision avoidance for other categories of vehicles, both manned and unmanned.

Electro-Optical System Analysis for Sense and Avoid

This paper presents a parametric analysis of the sense and avoid capability for an electrooptical system on unmanned aircraft. Our sensor analysis is based on simulated encounters from a new U.S. airspace encounter model that provides a comprehensive distribution of typical visual ight rule (VFR) aircraft behavior and encounter geometries. We assess the exchange between the sensor eld-of-view shape and detection range with the probability of intruder detection prior to near miss. This assessment also includes a trade-o analysis between eld-of-view azimuth angle and probability of detection with xed tracking technology (i.e. pixel array sensor and tracking algorithm). Initial results suggest that current standards are suitable for detecting larger aircraft but may not be ideal for small aircraft such as ultralights.

Hazard Alerting using Line-of-Sight Rate

This paper presents an analysis of an electro-optical hazard alerting system based on intruder line-of-sight rate. We use a recently-developed airspace encounter model to analyze intruder line-of-sight rate behavior prior to near miss. We look at a simple hazard alerting system that alerts whenever the line-of-sight rate drops below some set threshold. Simulations demonstrate that such an approach, regardless of the chosen threshold, leads to frequent false alerts. We explain how the problem of hazard alerting can also be formulated as a partially observable Markov decision process (POMDP) and show how such an approach signicantly decreases the false alert rate.