Zhaodan Kong

A Survey of Motion Planning Algorithms from the Perspective of Autonomous UAV Guidance

A fundamental aspect of autonomous vehicle guidance is planning trajectories. Historically, two fields have contributed to trajectory or motion planning methods: robotics and dynamics and control. The former typically have a stronger focus on computational issues and real-time robot control, while the latter emphasize the dynamic behavior and more specific aspects of trajectory performance. Guidance for Unmanned Aerial Vehicles (UAVs), including fixed- and rotary-wing aircraft, involves significant differences from most traditionally defined mobile and manipulator robots. Qualities characteristic to UAVs include non-trivial dynamics, three-dimensional environments, disturbed operating conditions, and high levels of uncertainty in state knowledge. Otherwise, UAV guidance shares qualities with typical robotic motion planning problems, including partial knowledge of the environment and tasks that can range from basic goal interception, which can be precisely specified, to more general tasks like surveillance and reconnaissance, which are harder to specify. These basic planning problems involve continual interaction with the environment. The purpose of this paper is to provide an overview of existing motion planning algorithms while adding perspectives and practical examples from UAV guidance approaches.

Anomaly detection in cyber-physical systems: A formal methods approach

As the complexity of cyber-physical systems increases, so does the number of ways an adversary can disrupt them. This necessitates automated anomaly detection methods to detect possible threats. In this paper, we extend our recent results in the field of inference via formal methods to develop an unsupervised learning algorithm. Our procedure constructs from data a signal temporal logic (STL) formula that describes normal system behavior. Trajectories that do not satisfy the learned formula are flagged as anomalous. STL can be used to formulate properties such as “If the train brakes within 500 m of the platform at a speed of 50 km/hr, then it will stop in at least 30 s and at most 50 s.” STL gives a more human-readable representation of behavior than classifiers represented as surfaces in high-dimensional feature spaces. STL formulae can also be used for early detection via online monitoring and for anomaly mitigation via formal synthesis. We demonstrate the power of our method with a physical model of a trains brake system. To our knowledge, this paper is the first instance of formal methods being applied to anomaly detection.

Modeling Human Guidance Behavior Based on Patterns in Agent–Environment Interactions

This paper presents the foundations for the analysis and modeling of human guidance behavior that is based on the emergent patterns in the closed-loop agent-environment dynamics. The central hypothesis is that these patterns, which can be explained in terms of invariants inherent to the closed-loop dynamics, provide the building blocks for the organization of human guidance behavior. The concept of interaction patterns is first introduced using a toy example and then detailed formally using dynamical system and control principles. This paper then demonstrates the existence and significance of interaction patterns in human guidance behavior that is based on data collected using guidance experiments with a miniature helicopter. The results confirm that human guidance behavior indeed exhibits invariances as defined by interaction patterns. The trajectories that are associated with each interaction pattern are then further decomposed by applying piecewise linear identification. The resulting elements are then combined under a hierarchical model that provides a natural and formal description of human guidance behavior.

Receding horizon trajectory optimization with a finite-state value function approximation

This paper describes a finite-horizon receding horizon trajectory optimization scheme which uses an approximation of the value function to provide cost-to-go (CTG) and associated state information. The value function approximation is computed using a finite-state, motion primitive automaton approximation of the vehicle dynamics. Using an actual value function approximation instead of heuristic CTG allows a tighter integration between the planning and control layers needed for vehicles operating in challenging spatial environments. It also enables a more rigorous use of the receding horizon control framework for autonomous control applications. The paper describes the finite-state value function approximation and its integration into the receding horizon scheme. Simulation examples illustrate the schemes capabilities and highlight interesting open issues that will need to be addressed to take full advantage of the approach.

Research Infrastructure for Interactive Human- and Autonomous Guidance

This paper describes a research infrastructure setup to exercise and investigate guidance and control capabilities under human and autonomous control modalities. The lab facility is designed to implement tasks that emphasize agent-environment interactions. The overall goal is to characterize these interactions and to apply the gained knowledge to determine interaction models. These can then be used to design guidance and control algorithms as well as human–machine systems. The facility uses miniature rotorcraft as test vehicles with a Vicon motion tracking system and SensoMotoric gaze tracking system. The facility also includes a high-fidelity simulation system to support larger scale autonomy and teleoperation experiments. The simulation incorporates the software components and models of the key flight hardware and sensors. The software system was integrated around the Robotics Operating System (ROS) to support the heterogenous processes and data and allow easy system reconfiguration. The paper describes the research objectives, details of the hardware and software components and their integration, and concludes with a summary of the ongoing research enabled by the lab facility including.

Mapping and Analysis of Human Guidance Performance From Trajectory Ensembles

This paper describes a mapping method for the analysis of guidance performance. Spatial state and time-to-go maps, along with their statistics, are computed from an ensemble of trajectories. The mapping technique is motivated by the concept of spatial value function associated with an optimal guidance model. For illustration, the method is applied to trajectories collected from a human-operated miniature helicopter in a precision interception task. The closed-loop dynamics of the helicopter under human control was modeled as a mass-point system. The closed-loop model provides a formal interpretation for the extracted maps and is also used to compute optimal trajectories that serve as absolute baseline for the guidance performance. The maps extracted from the experimental trajectories show that human guidance performance is sufficiently stationary and spatially coherent to be meaningfully embedded in a spatial map. The comparison with the optimal baseline makes it possible to identify the subjects specific performance gaps. Performance metrics that are defined and computed on hand of the maps enable more detailed assessment of the operators performance. The general results demonstrate that the guidance performance of a trained subject can be meaningfully modeled as a guidance policy based on a simple closed-loop mass-point model.

Mapping 3D Guidance Performance Using Approximate Optimal Cost-to-go Function

Being able to define and measure the performance of guidance systems is fundamental to their proper development. This task is particularly challenging for unmanned aerial vehicles operating in complex spatial environment like cities or mountains, both of which are operational theaters of predilection. Previous research mainly focuses on relative performance metrics. This paper, introduces a framework to derive absolute metrics. The approach is based on the idea that many guidance problems have a meaningful formulation as an optimal control problem. Hence the idea is that absolute performance metrics can be based on an approximation of the optimal control problem. In the following we approximate the vehicle dynamics using motion primitives. The choice of motion primitives is based on an analysis of the vehicle flight performance based on experimental data. As performance metric, we use the approximate cost-to-go maps and associated optimal states computed via dynamic programming for a cell-based world representation. The cost-to-go map, which can be computed for any environment described by Digital Terrain Elevation Data (DTED), provides a comprehensive insight into the interrelation of space, optimal behavior, as driven by a user specified performance index. The only limitation in the choice of performance index is that it should be a function of attributes of the finite state motion primitives. The paper illustrates the method applied to an unmanned R-MAX helicopter (10ft rotor diameter) operating in the down town of San Francisco. Three performance indexes are analyzed: time, energy and length.

An investigation of spatial behavior in agile guidance tasks

Humans are capable of agile and adaptive spatial behaviors that are far beyond the capabilities of todays autonomous systems. Spatial behaviors have been investigated in AI, cognitive science and neuroscience. This paper describes an engineering-oriented perspective that is intended to bridge the gap between these fields. We claim that spatial behavior cannot be fully understood by considering the agent and its environment separately. In our approach, we put the emphasis on the interaction between the agents dynamics and the task environment. We hypothesize that specific patterns emerge from this interaction and that these patterns are used by human operators to mitigate the complexity involved in agile and adaptive spatial performance. This paper describes preliminary experiments and a methodology to investigate these hypotheses.

Q-Learning for robust satisfaction of signal temporal logic specifications

This paper addresses the problem of learning optimal policies for satisfying signal temporal logic (STL) specifications by agents with unknown stochastic dynamics. The system is modeled as a Markov decision process, in which the states represent partitions of a continuous space and the transition probabilities are unknown. We formulate two synthesis problems where the desired STL specification is enforced by maximizing the probability of satisfaction, and the expected robustness degree, that is, a measure quantifying the quality of satisfaction. We discuss that Q-learning is not directly applicable to these problems because, based on the quantitative semantics of STL, the probability of satisfaction and expected robustness degree are not in the standard objective form of Q-learning. To resolve this issue, we propose an approximation of STL synthesis problems that can be solved via Q-learning, and we derive some performance bounds for the policies obtained by the approximate approach. The performance of the proposed method is demonstrated via simulations.

Perceptual modalities guiding bat flight in a native habitat

Flying animals accomplish high-speed navigation through fields of obstacles using a suite of sensory modalities that blend spatial memory with input from vision, tactile sensing, and, in the case of most bats and some other animals, echolocation. Although a good deal of previous research has been focused on the role of individual modes of sensing in animal locomotion, our understanding of sensory integration and the interplay among modalities is still meager. To understand how bats integrate sensory input from echolocation, vision, and spatial memory, we conducted an experiment in which bats flying in their natural habitat were challenged over the course of several evening emergences with a novel obstacle placed in their flight path. Our analysis of reconstructed flight data suggests that vision, echolocation, and spatial memory together with the possible exercise of an ability in using predictive navigation are mutually reinforcing aspects of a composite perceptual system that guides flight. Together with the recent development in robotics, our paper points to the possible interpretation that while each stream of sensory information plays an important role in bat navigation, it is the emergent effects of combining modalities that enable bats to fly through complex spaces.