Eric M. Wolff

Robust control of uncertain Markov Decision Processes with temporal logic specifications

We present a method for designing a robust control policy for an uncertain system subject to temporal logic specifications. The system is modeled as a finite Markov Decision Process (MDP) whose transition probabilities are not exactly known but are known to belong to a given uncertainty set. A robust control policy is generated for the MDP that maximizes the worst-case probability of satisfying the specification over all transition probabilities in this uncertainty set. To this end, we use a procedure from probabilistic model checking to combine the system model with an automaton representing the specification. This new MDP is then transformed into an equivalent form that satisfies assumptions for stochastic shortest path dynamic programming. A robust version of dynamic programming solves for a ε-suboptimal robust control policy with time complexity O(log1/ε) times that for the non-robust case.

Optimization-based trajectory generation with linear temporal logic specifications

We present a mathematical programming-based method for optimal control of discrete-time dynamical systems subject to temporal logic task specifications. We use linear temporal logic (LTL) to specify a wide range of properties and tasks, such as safety, progress, response, surveillance, repeated assembly, and environmental monitoring. Our method directly encodes an LTL formula as mixed-integer linear constraints on the continuous system variables, avoiding the computationally expensive processes of creating a finite abstraction of the system and a Büchi automaton for the specification. In numerical experiments, we solve temporal logic motion planning tasks for high-dimensional (10+ continuous state) dynamical systems.

Efficient reactive controller synthesis for a fragment of linear temporal logic

Motivated by robotic motion planning, we develop a framework for control policy synthesis for both non-deterministic transition systems and Markov decision processes that are subject to temporal logic task specifications. We introduce a fragment of linear temporal logic that can be used to specify common motion planning tasks such as safe navigation, response to the environment, persistent coverage, and surveillance. This fragment is computationally efficient; the complexity of control policy synthesis is a doubly-exponential improvement over standard linear temporal logic for both non-deterministic transition systems and Markov decision processes. This improvement is possible because we compute directly on the original system, as opposed to the automata-based approach commonly used. We give simulation results for representative motion planning tasks and compare to generalized reactivity(1).

Automaton-guided controller synthesis for nonlinear systems with temporal logic

We develop a method for the control of discrete-time nonlinear systems subject to temporal logic specifications. Our approach uses a coarse abstraction of the system and an automaton representing the temporal logic specification to guide the search for a feasible trajectory. This decomposes the search for a feasible trajectory into a series of constrained reachability problems. Thus, one can create controllers for any system for which techniques exist to compute (approximate) solutions to constrained reachability problems. Representative techniques include sampling-based methods for motion planning, reachable set computations for linear systems, and graph search for finite discrete systems. Our approach avoids the expensive computation of a discrete abstraction, and its implementation is amenable to parallel computing. We demonstrate our approach with numerical experiments on temporal logic motion planning problems with high-dimensional (10+ states) continuous systems.

Optimal Control with Weighted Average Costs and Temporal Logic Specifications

We consider optimal control for a system subject to temporal logic constraints. We minimize a weighted average cost function that generalizes the commonly used average cost function from discrete-time optimal control. Dynamic programming algorithms are used to construct an optimal trajectory for the system that minimizes the cost function while satisfying a temporal logic specification. Constructing an optimal trajectory takes only polynomially more time than constructing a feasible trajectory. We demonstrate our methods on simulations of autonomous driving and robotic surveillance tasks.

Optimal control of non-deterministic systems for a computationally efficient fragment of temporal logic

We develop a framework for optimal control policy synthesis for non-deterministic transition systems subject to temporal logic specifications. We use a fragment of temporal logic to specify tasks such as safe navigation, response to the environment, persistence, and surveillance. By restricting specifications to this fragment, we avoid a potentially doubly-exponential automaton construction. We compute feasible control policies for non-deterministic transition systems in time polynomial in the size of the system and specification. We also compute optimal control policies for average, minimax (bottleneck), and average cost-per-task-cycle cost functions. We highlight several interesting cases when optimal policies can be computed in time polynomial in the size of the system and specification. Additionally, we make connections between computing optimal control policies for an average cost-per-task-cycle cost function and the generalized traveling salesman problem. We give simulation results for motion planning problems.

Optimal Control of Nonlinear Systems with Temporal Logic Specifications

We present a mathematical programming-based method for optimal control of nonlinear systems subject to temporal logic task specifications. We specify tasks using a fragment of linear temporal logic (LTL) that allows both finite- and infinite-horizon properties to be specified, including tasks such as surveillance, periodic motion, repeated assembly, and environmental monitoring. Our method directly encodes an LTL formula as mixed-integer linear constraints on the system variables, avoiding the computationally expensive process of creating a finite abstraction. Our approach is efficient; for common tasks our formulation uses significantly fewer binary variables than related approaches and gives the tightest possible convex relaxation. We apply our method on piecewise affine systems and certain classes of differentially flat systems. In numerical experiments, we solve temporal logic motion planning tasks for high-dimensional (10\(+\) continuous state) systems.

Cross-entropy temporal logic motion planning

This paper presents a method for optimal trajectory generation for discrete-time nonlinear systems with linear temporal logic (LTL) task specifications. Our approach is based on recent advances in stochastic optimization algorithms for optimal trajectory generation. These methods rely on estimation of the rare event of sampling optimal trajectories, which is achieved by incrementally improving a sampling distribution so as to minimize the cross-entropy. A key component of these stochastic optimization algorithms is determining whether or not a trajectory is collision-free. We generalize this collision checking to efficiently verify whether or not a trajectory satisfies a LTL formula. Interestingly, this verification can be done in time polynomial in the length of the LTL formula and the trajectory. We also propose a method for efficiently re-using parts of trajectories that only partially satisfy the specification, instead of simply discarding the entire sample. Our approach is demonstrated through numerical experiments involving Dubins car and a generic point-mass model subject to complex temporal logic task specifications.

A Compositional Approach to Stochastic Optimal Control with Co-safe Temporal Logic Specifications

We introduce an algorithm for the optimal control of stochastic nonlinear systems subject to temporal logic constraints on their behavior. We compute directly on the state space of the system, avoiding the expensive pre-computation of a discrete abstraction. An automaton that corresponds to the temporal logic specification guides the computation of a control policy that maximizes the probability that the system satisfies the specification. This reduces controller synthesis to solving a sequence of stochastic constrained reachability problems. Each individual reachability problem is solved via the Hamilton-Jacobi-Bellman (HJB) partial differential equation of stochastic optimal control theory. To increase the efficiency of our approach, we exploit a class of systems where the HJB equation is linear due to structural assumptions on the noise. The linearity of the partial differential equation allows us to pre-compute control policy primitives and then compose them, at essentially zero cost, to conservatively satisfy a complex temporal logic specification.

Efficient control synthesis for augmented finite transition systems with an application to switching protocols

Augmented finite transition systems generalize nondeterministic transition systems with additional liveness conditions. We propose efficient algorithms for synthesizing control protocols for augmented finite transition systems to satisfy high-level specifications expressed in a fragment of linear temporal logic (LTL). We then use these algorithms within a framework for switching protocol synthesis for discrete-time dynamical systems, where augmented finite transition systems are used for abstracting the underlying dynamics. We introduce a notion of minimality for abstractions of certain fidelity and show that such minimal abstractions can be exactly computed for switched affine systems. Additionally, based on this framework, we present a procedure for computing digitally implementable switching protocols for continuous-time systems. The effectiveness of the proposed framework is illustrated through two examples of temperature control for buildings.