Jerry Ding

Guaranteed decentralized pursuit-evasion in the plane with multiple pursuers

Pursuit-evasion games are an important problem in robotics and control, but games with many players are difficult to analyze and solve. This paper studies a game of multiple pursuers cooperating to capture a single evader in a bounded, convex, polytope in the plane. We present a decentralized control scheme based on the Voronoi partion of the game domain, where the pursuers jointly minimize the area of the evaders Voronoi cell. We prove that capturing the evader is guaranteed under this scheme regardless of the evaders actions, and show simulation results demonstrating the pursuit strategy.

Reachability calculations for automated aerial refueling

This paper describes Hamilton-Jacobi (HJ) reachability calculations for a hybrid systems formalism governing unmanned aerial vehicles (UAVs) interacting with another vehicle in a safety-critical situation. We use this problem to lay the foundations toward the goal of refining or designing protocols for multi-UAV and/or manned vehicle interaction. We describe here what mathematical foundations are necessary to formulate verification problems on reachability and safety of flight maneuvers. We finally show how this formalism can be used in the chosen application to inform UAV decisions on avoiding unsafe scenarios while achieving mission objectives.

A differential game approach to planning in adversarial scenarios: A case study on capture-the-flag

Capture-the-flag is a complex, challenging game that is a useful proxy for many problems in robotics and other application areas. The game is adversarial, with multiple, potentially competing, objectives. This interplay between different factors makes the problem complex, even in the case of only two players. To make analysis tractable, previous approaches often make various limiting assumptions upon player actions. In this paper, we present a framework for analyzing and solving a two-player capture-the-flag game as a zero-sum differential game. Our problem formulation allows each player to make decisions rationally based upon the current player positions, assuming only an upper bound on the movement speeds. Using Hamilton-Jacobi reachability analysis, we compute winning regions for each player as subsets of the joint configuration space and derive the corresponding winning strategies. Simulation results are presented along with implications of the work as a tool for automation-aided decision-making for humans and mixed human-robot teams.

Reachability-based synthesis of feedback policies for motion planning under bounded disturbances

The task of planning and controlling robot motion in practical applications is often complicated by the effects of model uncertainties and environment disturbances. We present in this paper a systematic approach for generating robust motion control strategies to satisfy high level specifications of safety, target attainability, and invariance, under unknown but bounded, continuous disturbances. The motion planning task is decomposed into the two sub-problems of finite horizon reach with avoid and infinite horizon invariance. The set of states for which each of the sub-problems is robustly feasible is computed via iterative reachability calculations under a differential game framework. We discuss how the results of this computation can be used to inform selections of control inputs based upon state measurements at run-time and provide an algorithm for implementing the corresponding feedback control policies. Finally, we demonstrate an experimental application of this method to the control of an autonomous helicopter in tracking a moving ground vehicle.

Time-optimal multi-stage motion planning with guaranteed collision avoidance via an open-loop game formulation

We present an efficient algorithm which computes, for a kinematic point mass moving in the plane, a time-optimal path that visits a sequence of target sets while conservatively avoiding collision with moving obstacles, also modelled as kinematic point masses, but whose trajectories are unknown. The problem is formulated as a pursuit-evasion differential game, and the underlying construction is based on optimal control. The algorithm, which is a variant of the fast marching method for shortest path problems, can handle general dynamical constraints on the players and arbitrary domain geometry (e.g. obstacles, non-polygonal boundaries). Applications to a two-stage game, capture-the-flag, is presented.

Robust reach-avoid controller synthesis for switched nonlinear systems

In this paper, we describe a method to automatically synthesize controllers that provide hard guarantees of safety and target reachability for sampled-data switched systems under bounded continuous disturbances. Techniques from hybrid system verification are used to perform continuous time differential game calculations on each sampling interval. Iterative procedures are given for computing the set of states for which there exists an admissible control policy so that the closed-loop system satisfies the properties of safety and reachability over a finite time horizon. From this computation, we show how to obtain an explicit state feedback policy in the form of multiple reachable sets, and an algorithm is given for using this feedback law in closed-loop control of the switched system. A simulation example of automated aerial refueling is used to illustrate the application of our approach.

A stochastic games framework for verification and control of discrete time stochastic hybrid systems

We describe a framework for analyzing probabilistic reachability and safety problems for discrete time stochastic hybrid systems within a dynamic games setting. In particular, we consider finite horizon zero-sum stochastic games in which a control has the objective of reaching a target set while avoiding an unsafe set in the hybrid state space, and a rational adversary has the opposing objective. We derive an algorithm for computing the maximal probability of achieving the control objective, subject to the worst-case adversary behavior. From this algorithm, sufficient conditions of optimality are also derived for the synthesis of optimal control policies and worst-case disturbance strategies. These results are then specialized to the safety problem, in which the control objective is to remain within a safe set. We illustrate our modeling framework and computational approach using both a tutorial example with jump Markov dynamics and a practical application in the domain of air traffic management.

Hybrid Systems in Robotics

Robotics has provided the motivation and inspiration for many innovations in planning and control. From nonholonomic motion planning [1] to probabilistic road maps [2], from capture basins [3] to preimages [4] of obstacles to avoid, and from geometric nonlinear control [5], [6] to machine-learning methods in robotic control [7], there is a wide range of planning and control algorithms and methodologies that can be traced back to a perceived need or anticipated benefit in autonomous or semiautonomous systems.

Optimal control of partially observable discrete time stochastic hybrid systems for safety specifications

This paper describes a theoretical framework for the design of controllers to satisfy probabilistic safety specifications for partially observable discrete time stochastic hybrid systems. We formulate the problem as a partial information stochastic optimal control problem, in which the objective is to maximize the probability that the state trajectory remains within a given safe set in the hybrid state space, using observations of the history of inputs and outputs. It is shown that this optimal control problem, which has a multiplicative payoff structure, is equivalent to a terminal payoff problem when the state space is augmented with a binary random variable capturing the safety of past state evolution. This allows us to derive a sufficient statistic for the probabilistic safety problem as a set of Bayesian filtering equations updating a conditional distribution on the augmented state space, as well as an abstract dynamic programming algorithm for computing the maximal probability of safety and an optimal control policy.

Automation-Assisted Capture-the-Flag: A Differential Game Approach

Capture-the-flag is a complex, challenging game that is a useful proxy for many problems in robotics and other application areas. The game is adversarial, with multiple, potentially competing, objectives. This interplay among different factors makes the problem complex, even in the case of only two players. To make analysis tractable, previous approaches often make various limiting assumptions upon player actions. In this paper, we present a framework for analyzing and solving a two-player capture-the-flag game as a zero-sum differential game. Our problem formulation allows each player to make decisions rationally according to the current player positions, assuming only an upper bound on the movement speeds. Using numerical solutions to Hamilton-Jacobi-Isaacs equations, we compute winning regions for each player as subsets of the joint configuration space and derive the corresponding winning strategies. The computational method and simulations are presented, along with experiments with human agents in the Berkeley autonomy and robotics in capture-the-flag testbed. These experiments demonstrate the use of the solutions in realistic conditions and highlight their potential applications in automation-aided decision making for humans and mixed human-robot teams.