Jung-Su Ha

A topology-guided path integral approach for stochastic optimal control

This work presents an efficient method to solve a class of continuous-time, continuous-space stochastic optimal control problems of robot motion in a cluttered environment. The method builds upon a path integral representation of the stochastic optimal control problem that allows computation of the optimal solution through sampling and estimation process. As this sampling process often leads to a local minimum especially when the state space is highly non-convex due to the obstacle field, we present an efficient method to alleviate this issue by devising a proposed topological motion planning algorithm. Combined with a receding-horizon scheme in execution of the optimal control solution, the proposed method can generate a dynamically feasible and collision-free trajectory while reducing concern about local optima. Illustrative numerical examples are presented to demonstrate the applicability and validity of the proposed approach.

A successive approximation-based approach for optimal kinodynamic motion planning with nonlinear differential constraints

This paper presents an extension to RRT* [1], a sampling-based motion planning with asymptotic optimality guarantee, in order to incorporate nonlinear differential equations in motion dynamics. The main challenge due to nonlinear differential constraints is the computational complexity of solving a two-point boundary-value problem that arises in the tree expansion process to optimally connect two given states. This work adapts the successive approximation method that transforms a nonlinear optimal control problem into a sequence of linear-quadratic-like problems to solve these TPBVPs. The resulting algorithm, termed SA-RRT*, is demonstrated to create more realistic plans compared to existing kinodynamic extensions of RRT*, while preserving asymptotic optimality.

Multiscale abstraction, planning and control using diffusion wavelets for stochastic optimal control problems

This work presents a multiscale framework to solve a class of stochastic optimal control problems in the context of robot motion planning and control in a complex environment. In order to handle complications resulting from a large decision space and complex environmental geometry, two key concepts are adopted: (a) a diffusion wavelet representation of the Markov chain for hierarchical abstraction of the state space; and (b) a desirability function-based representation of the Markov decision process (MDP) to efficiently calculate the optimal policy. In the proposed framework, a global plan that compressively takes into account the long time/length-scale state transition is first obtained by approximately solving an MDP whose desirability function is represented by coarse scale bases in the hierarchical abstraction. Then, a detailed local plan is computed by solving an MDP that considers wavelet bases associated with a focused region of the state space, guided by the global plan. The resulting multiscale plan is utilized to finally compute a continuous-time optimal control policy within a receding horizon implementation. Two numerical examples are presented to demonstrate the applicability and validity of the proposed approach.

Informative windowed forecasting of continuous-time linear systems for mutual information-based sensor planning

This paper presents an expression of mutual information that defines the information gain in planning of sensing resources, when the goal is to reduce the forecast uncertainty of some quantities of interest and the system dynamics is described as a continuous-time linear system. The method extends the smoother approach in Choi and How (2010b) to handle a more general notion of the verification entity-continuous sequence of variables over some finite time window in the future. The expression of mutual information for this windowed forecasting case is derived and quantified, taking advantage of an underlying conditional independence structure and utilizing a two-filter formula for fixed-interval smoothing with correlated noises. Two numerical examples on (a) a two-state linear system with time-varying one-way coupling dynamics, and (b) idealized weather forecasting with moving verification paths demonstrate the validity of the proposed quantification methodology.

Informative Path Planning and Mapping with Multiple UAVs in Wind Fields

Informative path planning (IPP) is used to design paths for robotic sensor platforms to extract the best/maximum possible information about a quantity of interest while operating under a set of constraints, such as dynamic feasibility of vehicles. The key challenges of IPP are the strong coupling in multiple layers of decisions: the selection of locations to visit, the allocation of sensor platforms to those locations; and the processing of the gathered information along the paths. This paper presents an systematic procedure for IPP and environmental mapping using multiple UAV sensor platforms. It (a) selects the best locations to observe, (b) calculates the cost and finds the best paths for each UAV, and (c) estimates the measurement value within a given region using the Gaussian process (GP) regression framework. An illustrative example of RF intensity field mapping is presented to demonstrate the validity and applicability of the proposed approach.

A stochastic game-theoretic approach for analysis of multiple cooperative air combat

This paper presents a game theory-based methodology to analyze multiple beyond-visual-range (BVR) air combat scenarios. The combat scenarios are formalized as a stochastic game consisting of a sequence of normal-form games with a continuous sub-game. The proposed game formulation improves the scalability by reducing the decision space while taking advantage of some underlying symmetry structures of the combat scenario. The equilibrium strategy and the value functions of the game are computed through some dynamic programming procedure; the impact of the aircrafts velocity and the cooperation scheme for the combat is analyzed based on the equilibrium strategies.

MDP-Based Mission Planning for Multi-UAV Persistent Surveillance

This paper presents a methodology to generate task flow for conducting a surveillance mission using multiple UAVs, when the goal is to persistently maintain the uncertainty level of surveillance regions as low as possible. The mission planning problem is formulated as a Markov decision process (MDP), which is a infinite-horizon discrete stochastic optimal control formulation and often leads to a periodic task flows to be implemented in a persistent manner. The method specifically focuses on reducing the size of decision space without losing key feature of the problem in order to mitigate the curse of dimensionality of MDP; integrating a task allocator to identify admissible actions is demonstrate to effectively reduce the decision space. Numerical simulations verify the applicability of the proposed decision scheme.

Iterative Methods for Efficient Sampling-Based Optimal Motion Planning of Nonlinear Systems

Abstract This paper extends the RRT* algorithm, a recently developed but widely used sampling based optimal motion planner, in order to effectively handle nonlinear kinodynamic constraints. Nonlinearity in kinodynamic differential constraints often leads to difficulties in choosing an appropriate distance metric and in computing optimized trajectory segments in tree construction. To tackle these two difficulties, this work adopts the affine quadratic regulator-based pseudo-metric as the distance measure and utilizes iterative two-point boundary value problem solvers to compute the optimized segments. The proposed extension then preserves the inherent asymptotic optimality of the RRT* framework, while efficiently handling a variety of kinodynamic constraints. Three numerical case studies validate the applicability of the proposed method.

Periodic sensing trajectory generation for persistent monitoring

This paper presents periodic trajectory optimization method for a mobile sensor performing persistent monitoring to maintain the uncertainty in the environment at the minimum. The uncertain environment is represented by a set of deterministic spatial basis function with the stochastic temporal dynamic coefficients. An optimal control problem is formulated to determine the optimal periodic trajectory of the sensor and the uncertainty state as well as the initial condition and the period. The path induces the periodic Riccati equation and is proven to lead an arbitrary initial uncertainty state to the optimized periodic trajectory. It is also shown that the resulting optimal periodic solution can be used to develop a subopitmal filtering mechanism for the mobile sensor. A simple synthetic example is presented for preliminary demonstration the validity of the proposed methodology, producing physically meaningful sensing trajectories.