Pingping Zhu

A Hybrid-Adaptive Dynamic Programming Approach for the Model-Free Control of Nonlinear Switched Systems

This paper presents a hybrid adaptive dynamic programming (hybrid-ADP) approach for determining the optimal continuous and discrete control laws of a switched system online, solely from state observations. The new hybrid-ADP recurrence relationships presented are applicable to model-free control of switched hybrid systems that are possibly nonlinear. The computational complexity and convergence of the hybrid-ADP approach are analyzed, and the method is validated numerically showing that the optimal controller and value function can be learned iteratively online from state observations.

A Mobile Sensing Approach for Regional Surveillance of Fugitive Methane Emissions in Oil and Gas Production

This paper addresses the need for surveillance of fugitive methane emissions over broad geographical regions. Most existing techniques suffer from being either extensive (but qualitative) or quantitative (but intensive with poor scalability). A total of two novel advancements are made here. First, a recursive Bayesian method is presented for probabilistically characterizing fugitive point-sources from mobile sensor data. This approach is made possible by a new cross-plume integrated dispersion formulation that overcomes much of the need for time-averaging concentration data. The method is tested here against a limited data set of controlled methane release and shown to perform well. We then present an information-theoretic approach to plan the paths of the sensor-equipped vehicle, where the path is chosen so as to maximize expected reduction in integrated target source rate uncertainty in the region, subject to given starting and ending positions and prevailing meteorological conditions. The information-driven sensor path planning algorithm is tested and shown to provide robust results across a wide range of conditions. An overall system concept is presented for optionally piggybacking of these techniques onto normal industry maintenance operations using sensor-equipped work trucks.

Optimized visibility motion planning for target tracking and localization

This paper presents a visibility-based method for planning the motion of a mobile robotic sensor with bounded field-of-view to optimally track a moving target while localizing itself. The target and robot states are estimated from online sensor measurements and a set of a priori known landmarks, using an extended Kalman filter (EKF), and thus the proposed method is applicable to robots without a global positioning system. It is shown that the problem of optimizing the target tracking and robot localization performance is equivalent to optimizing the visibility or probability of detection in the EKF framework under mild assumptions. The control law that maximizes the probability of detection for a robotic sensor with a sector-shaped field-of-view (FoV) is derived as a function of the robot heading and aperture. Simulations have been conducted on synthetic experiments and the results show that the optimized-visibility approach is effective at avoiding target loss, and outperforms a state-of-the-art potential method based on robot trailer models [1].

Distributed Optimal Control of Multiscale Dynamical Systems: A Tutorial

Many complex systems, ranging from renewable resources [1] to very-large-scale robotic systems (VLRS) [2], can be described as multiscale dynamical systems comprising many interactive agents. In recent years, significant progress has been made in the formation control and stability analysis of teams of agents, such as robots, or autonomous vehicles. In these systems, the mutual goals of the agents are, for example, to maintain a desired configuration, such as a triangle or a star formation, or to perform a desired behavior, such as translating as a group (schooling) or maintaining the center of mass of the group (flocking) [2]-[7]. While this literature has successfully illustrated that the behavior of large networks of interacting agents can be conveniently described and controlled by density functions, it has yet to provide an approach for optimizing the agent density functions such that their mutual goals are optimized.

Camera control for learning nonlinear target dynamics via Bayesian nonparametric Dirichlet-process Gaussian-process (DP-GP) models

This paper presents a camera control approach for learning unknown nonlinear target dynamics by approximating information value functions using particles that represent targets position distributions. The target dynamics are described by a non-parametric mixture model that can learn a potentially infinite number of motion patterns. Assuming that each motion pattern can be represented as a velocity field, the target behaviors can be described by a non-parametric Dirichlet process-Gaussian process (DP-GP) mixture model. The DP-GP model has been successfully applied for clustering time-invariant spatial phenomena due to its flexibility to adapt to data complexity without overfitting. A new DP-GP information value function is presented that can be used by the sensor to explore and improve the DP-GP mixture model. The optimal camera control is computed to maximize this information value function online via a computationally efficient particle-based search method. The proposed approach is demonstrated through numerical simulations and hardware experiments in the RAVEN testbed at MIT.

Distributed Optimal Control of Sensor Networks for Dynamic Target Tracking

This paper presents a distributed optimal control approach for managing omnidirectional sensor networks deployed to cooperatively track moving targets in a region of interest. Several authors have shown that under proper assumptions, the performance of mobile sensors is a function of the sensor distribution. In particular, the probability of cooperative track detection, also known as track coverage, can be shown to be an integral function of a probability density function representing the macroscopic sensor network state. Thus, a mobile sensor network deployed to detect moving targets can be viewed as a multiscale dynamical system in which a time-varying probability density function can be identified as a restriction operator, and optimized subject to macroscopic dynamics represented by the advection equation. Simulation results show that the distributed control approach is capable of planning the motion of hundreds of cooperative sensors, such that their effectiveness is significantly increased compared to that of existing uniform, grid, random, and stochastic gradient methods.

Value function approximation for the control of multiscale dynamical systems

This paper presents a novel approximate dynamic programming (ADP) algorithm for the optimal control of multiscale dynamical systems comprised of many interacting agents. The ADP algorithm presented in this paper is obtained using a distributed optimal control approach by which the performance of the multiscale dynamical system is represented in terms of a macroscopic state, and is optimized subject to a macroscopic description provided by the continuity equation. A value function approximation scheme is proposed and tested using a data set obtained by solving the necessary conditions for optimality for the distributed optimal control problem. The results shows that the proposed approximation method can learn the value function accurately and, thus, may be applied to adapt the optimal control law.

Multi-kernel probability distribution regressions

This paper presents a multi-layer reproducing kernel Hilbert space (RKHS) approach for probability distribution to real and probability distribution to function regressions. The approach maps the distributions into RKHS by distribution embeddings and, then, constructs a multi-layer RKHS within which the multi-kernel distribution regression can be implemented using an existing kernel regression algorithm, such as kernel recursive least squares (KRLS). The numerical simulations on synthetic data obtained via Gaussian mixtures show that the proposed approach outperforms existing probability distribution (DR) regression algorithms by achieving smaller mean squared errors (MSEs) and requiring less training samples.

Confidence level estimation in multi-target classification problems

This paper presents an approach for estimating the confidence level in automatic multi-target classification performed by an imaging sensor on an unmanned vehicle. An automatic target recognition algorithm comprised of a deep convolutional neural network in series with a support vector machine classifier detects and classifies targets based on the image matrix. The joint posterior probability mass function of target class, features, and classification estimates is learned from labeled data, and recursively updated as additional images become available. Based on the learned joint probability mass function, the approach presented in this paper predicts the expected confidence level of future target classifications, prior to obtaining new images. The proposed approach is tested with a set of simulated sonar image data. The numerical results show that the estimated confidence level provides a close approximation to the actual confidence level value determined a posteriori, i.e. after the new image is obtained by the on-board sensor. Therefore, the expected confidence level function presented in this paper can be used to adaptively plan the path of the unmanned vehicle so as to optimize the expected confidence levels and ensure that all targets are classified with satisfactory confidence after the path is executed.

A Generalized Reduced Gradient Method for the Optimal Control of Very-Large-Scale Robotic Systems

This paper develops a new indirect method for distributed optimal control (DOC) that is applicable to optimal planning for very-large-scale robotic (VLSR) systems in complex environments. The method is inspired by the nested analysis and design method known as generalized reduced gradient (GRG). The computational complexity analysis presented in this paper shows that the GRG method is significantly more efficient than classical optimal control or direct DOC methods. The GRG method is demonstrated for VLSR path planning in obstacle-populated environments in which robots are subject to external forces and disturbances. The results show that the method significantly improves performance compared to the existing direct DOC and stochastic gradient methods.