Jaime F. Fisac

Reachability-based safe learning with Gaussian processes

Reinforcement learning for robotic applications faces the challenge of constraint satisfaction, which currently impedes its application to safety critical systems. Recent approaches successfully introduce safety based on reachability analysis, determining a safe region of the state space where the system can operate. However, overly constraining the freedom of the system can negatively affect performance, while attempting to learn less conservative safety constraints might fail to preserve safety if the learned constraints are inaccurate. We propose a novel method that uses a principled approach to learn the systems unknown dynamics based on a Gaussian process model and iteratively approximates the maximal safe set. A modified control strategy based on real-time model validation preserves safety under weaker conditions than current approaches. Our framework further incorporates safety into the reinforcement learning performance metric, allowing a better integration of safety and learning. We demonstrate our algorithm on simulations of a cart-pole system and on an experimental quadrotor application and show how our proposed scheme succeeds in preserving safety where current approaches fail to avoid an unsafe condition.

Reach-avoid problems with time-varying dynamics, targets and constraints

We consider a reach-avoid differential game, in which one of the players aims to steer the system into a target set without violating a set of state constraints, while the other player tries to prevent the first from succeeding; the system dynamics, target set, and state constraints may all be time-varying. The analysis of this problem plays an important role in collision avoidance, motion planning and aircraft control, among other applications. Previous methods for computing the guaranteed winning initial conditions and strategies for each player have either required augmenting the state vector to include time, or have been limited to problems with either no state constraints or entirely static targets, constraints and dynamics. To incorporate time-varying dynamics, targets and constraints without the need for state augmentation, we propose a modified Hamilton-Jacobi-Isaacs equation in the form of a double-obstacle variational inequality, and prove that the zero sublevel set of its viscosity solution characterizes the capture basin for the target under the state constraints. Through this formulation, our method can compute the capture basin and winning strategies for time-varying games at virtually no additional computational cost relative to the time-invariant case. We provide an implementation of this method based on well-known numerical schemes and show its convergence through a simple example; we include a second example in which our method substantially outperforms the state augmentation approach.

Safe sequential path planning of multi-vehicle systems via double-obstacle Hamilton-Jacobi-Isaacs variational inequality

We consider the problem of planning trajectories for a group of N vehicles, each aiming to reach its own target set while avoiding danger zones of other vehicles. The analysis of problems like this is extremely important practically, especially given the growing interest in utilizing unmanned aircraft systems for civil purposes. The direct solution of this problem by solving a single-obstacle Hamilton-Jacobi-Isaacs (HJI) variational inequality (VI) is numerically intractable due to the exponential scaling of computation complexity with problem dimensionality. Furthermore, the single-obstacle HJI VI cannot directly handle situations in which vehicles do not have a common scheduled arrival time. Instead, we perform sequential path planning by considering vehicles in order of priority, modeling higher-priority vehicles as time-varying obstacles for lower-priority vehicles. To do this, we solve a double-obstacle HJI VI which allows us to obtain the reach-avoid set, defined as the set of states from which a vehicle can reach its target while staying within a time-varying state constraint set. From the solution of the double-obstacle HJI VI, we can also extract the latest start time and the optimal control for each vehicle. This is a first application of the double-obstacle HJI VI which can handle systems with time-varying dynamics, target sets, and state constraint sets, and results in computation complexity that scales linearly, as opposed to exponentially, with the number of vehicles in consideration.

Safe platooning of unmanned aerial vehicles via reachability

Recently, there has been immense interest in using unmanned aerial vehicles (UAVs) for civilian operations such as package delivery, firefighting, and fast disaster response. As a result, UAV traffic management systems are needed to support potentially thousands of UAVs flying simultaneously in the airspace, in order to ensure their liveness and safety requirements are met. Hamilton-Jacobi (HJ) reachability is a powerful framework for providing conditions under which these requirements can be met, and for synthesizing the optimal controller for meeting them. However, due to the curse of dimensionality, HJ reachability is only tractable for a small number of vehicles if their set of maneuvers is unrestricted. In this paper, we define a platoon to be a group of UAVs in a single-file formation. We model each vehicle as a hybrid system with modes corresponding to its role in the platoon, and specify the set of allowed maneuvers in each mode to make the analysis tractable. We propose several liveness controllers based on HJ reachability, and wrap a safety controller, also based on HJ reachability, around the liveness controllers. For a single altitude range, our approach guarantees safety for one safety breach; in the unlikely event of multiple safety breaches, safety can be guaranteed over multiple altitude ranges. We demonstrate the satisfaction of liveness and safety requirements through simulations of three common scenarios.

The pursuit-evasion-defense differential game in dynamic constrained environments

Dynamic multi-player games are powerful abstractions of important real-world problems involving multiple interacting agents in both cooperative and adversarial settings. This paper studies a three-player differential pursuit-evasion game in which a pursuer aims to capture a fleeing evader while a third player, the defender, cooperates with the latter by attempting to intercept or delay the pursuer to avoid capture. Our analysis considers time-varying dynamics and allows the presence of possibly moving obstacles in the domain. We apply a recent theoretical result to express the outcome of the game through the solution of a double-obstacle Hamilton-Jacobi-Isaacs variational inequality, and propose a novel approach to break down the problem into two simpler two-player games with dynamic targets and constraints, which can be solved at a much lower cost. Although conservative, this method guarantees correctness of the computed winning region and strategy for the evader-defender team when a feasible escape solution is found. We demonstrate both the full solution and the approximation method through a numerical example.

FailSafe: A generalized methodology for converter fault detection, identification, and remediation in nanogrids

We present the design, implementation, and experimental validation of FailSafe - a generalized methodology for fault detection, identification, and remediation (FDIR) for switching power converters in nanogrids. FailSafe is a dynamical systems approach to FDIR for switching power converters, and can be applied to a broad class of converters and fault types. FailSafe operates as part of the control loop of a switching power converter, and uses the measurements and inputs of the converter to achieve both fault detection and identification (FDI) and fault remediation. In this paper, we present two Modules for FDI - a model-based residual approach and a data-driven multiclass Support Vector Machine (one-vs-one) approach. Moreover, we describe the design of a fault remediation Module by designing optimal control actions in a pre-computed reach-avoid set. We present simulation and experimental results using a prototype nanogrid testbed. Simulation results for the multiclass Support Vector Machine (one-vs-one) FDI Module on a 6-phase interleaved boost converter demonstrate fault detection and identification with a classification accuracy of 98.9% for a current sensor fault and 90.8% for an output capacitor fault. Experimental results for the model-based residual FDI Module on a boost converter demonstrate fault detection and identification in 600 μs for a capacitor fault and 250 μs for a voltage sensor fault.