Austin Jones

Anomaly detection in cyber-physical systems: A formal methods approach

As the complexity of cyber-physical systems increases, so does the number of ways an adversary can disrupt them. This necessitates automated anomaly detection methods to detect possible threats. In this paper, we extend our recent results in the field of inference via formal methods to develop an unsupervised learning algorithm. Our procedure constructs from data a signal temporal logic (STL) formula that describes normal system behavior. Trajectories that do not satisfy the learned formula are flagged as anomalous. STL can be used to formulate properties such as “If the train brakes within 500 m of the platform at a speed of 50 km/hr, then it will stop in at least 30 s and at most 50 s.” STL gives a more human-readable representation of behavior than classifiers represented as surfaces in high-dimensional feature spaces. STL formulae can also be used for early detection via online monitoring and for anomaly mitigation via formal synthesis. We demonstrate the power of our method with a physical model of a trains brake system. To our knowledge, this paper is the first instance of formal methods being applied to anomaly detection.

Temporal Logics for Learning and Detection of Anomalous Behavior

The increased complexity of modern systems necessitates automated anomaly detection methods to detect possible anomalous behavior determined by malfunctions or external attacks. We present formal methods for inferring (via supervised learning) and detecting (via unsupervised learning) anomalous behavior. Our procedures use data to construct a signal temporal logic (STL) formula that describes normal system behavior. This logic can be used to formulate properties such as “If the train brakes within 500 m of the platform at a speed of 50 km/hr, then it will stop in at least 30 s and at most 50 s.” Our procedure infers not only the physical parameters involved in the formula (e.g., 500 m in the example above) but also its logical structure. STL gives a more human-readable representation of behavior than classifiers represented as surfaces in high-dimensional feature spaces. The learned formula enables us to perform early detection by using monitoring techniques and anomaly mitigation by using formal synthesis techniques. We demonstrate the power of our methods with examples of naval surveillance and a train braking system.

A receding horizon algorithm for informative path planning with temporal logic constraints

This paper considers the problem of finding the most informative path for a sensing robot under temporal logic constraints, a richer set of constraints than have previously been considered in information gathering. An algorithm for informative path planning is presented that leverages tools from information theory and formal control synthesis, and is proven to give a path that satisfies the given temporal logic constraints. The algorithm uses a receding horizon approach in order to provide a reactive, on-line solution while mitigating computational complexity. Statistics compiled from multiple simulation studies indicate that this algorithm performs better than a baseline exhaustive search approach.

Q-Learning for robust satisfaction of signal temporal logic specifications

This paper addresses the problem of learning optimal policies for satisfying signal temporal logic (STL) specifications by agents with unknown stochastic dynamics. The system is modeled as a Markov decision process, in which the states represent partitions of a continuous space and the transition probabilities are unknown. We formulate two synthesis problems where the desired STL specification is enforced by maximizing the probability of satisfaction, and the expected robustness degree, that is, a measure quantifying the quality of satisfaction. We discuss that Q-learning is not directly applicable to these problems because, based on the quantitative semantics of STL, the probability of satisfaction and expected robustness degree are not in the standard objective form of Q-learning. To resolve this issue, we propose an approximation of STL synthesis problems that can be solved via Q-learning, and we derive some performance bounds for the policies obtained by the approximate approach. The performance of the proposed method is demonstrated via simulations.

Distributed information gathering policies under temporal logic constraints

We present an algorithm for synthesizing distributed control policies for networks of mobile robots such that they gather the maximum amount of information about some a priori unknown feature of the environment, e.g. hydration levels of crops or a lost person adrift at sea. Natural motion and communication constraints such as “Avoid obstacles and periodically communicate with all other agents”, are formulated as temporal logic formulae, a richer set of constraints than has been previously considered for this application. Mission constraints are distributed automatically among sub-groups of the agents. Each sub-group independently executes a receding horizon planner that locally optimizes information gathering and is guaranteed to satisfy the assigned mission specification. This approach allows the agents to disperse beyond inter-agent communication ranges while ensuring global team constraints are met. We evaluate our novel paradigm via simulation.

Optimal pesticide scheduling in precision agriculture

Agricultural automation presents challenges typically encountered in the realm of cyber-physical systems, such as incomplete information (plant health indicators), external disturbances (weather), limited control authority (fertilizers cannot make a plant mature arbitrarily fast), and a combination of discrete events and continuous plant dynamics. In this paper, we investigate the problem of optimal pesticide spray scheduling. Regulations impose strict requirements on scheduling, e.g., individual pesticides are only effective during certain seasons and pesticides cannot be sprayed too close to harvest time. We show how to translate these requirements to a metric temporal logic formula over the space of schedules. We next use the theory of optimal mode scheduling to generate a schedule that minimizes the risk of various infestations over time while guaranteeing the satisfaction of the constraints. We demonstrate this methodology via simulation with scheduling constraints based on recommendations and regulations from agricultural experts. Our case study considers blueberries, a crop whose cultivation currently involves little automation.

A motion-based communication system

For some applications in team robotics, a wireless electronic communication system is not ideal. We propose for some of these tasks that it is more appropriate to communicate through motion, that is by encoding symbols in locomotion and decoding symbols using sensor data. We discuss some of the challenges and requirements of such a system and derive for the LTI case control policies used to enact trajectories that optimize a joint expression of control energy and robustness to observation noise.

Control in belief space with Temporal Logic specifications

In this paper, we present a sampling-based algorithm to synthesize control policies with temporal and uncertainty constraints. We introduce a specification language called Gaussian Distribution Temporal Logic (GDTL), an extension of Boolean logic that allows us to incorporate temporal evolution and noise mitigation directly into the task specifications, e.g. “Go to region A and reduce the variance of your state estimate below 0.1 m2.” Our algorithm generates a transition system in the belief space and uses local feedback controllers to break the curse of history associated with belief space planning. Furthermore, conventional automata-based methods become tractable. Switching control policies are then computed using a product Markov Decision Process (MDP) between the transition system and the Rabin automaton encoding the task specification. We present algorithms to translate a GDTL formula to a Rabin automaton and to efficiently construct the product MDP by leveraging recent results from incremental computing. Our approach is evaluated in hardware experiments using a camera network and ground robot.

Information-guided persistent monitoring under temporal logic constraints

We study the problem of planning the motion of an agent such that it maintains indefinitely a high-quality estimate of some a priori unknown feature, such as traffic levels in an urban environment. Persistent operation requires that the agent satisfy motion constraints, such as visiting charging stations infinitely often, which are readily described by rich linear temporal logic (LTL) specifications. We propose and evaluate via simulation a two-level dynamic programming algorithm that is guaranteed to satisfy given LTL constraints. The low-level path planner implements a receding horizon algorithm that maximizes the local information gathering rate. The high-level planner selects inputs to the low-level planner based on global performance considerations.

Distribution temporal logic: Combining correctness with quality of estimation

We present a new temporal logic called Distribution Temporal Logic (DTL) defined over predicates of belief states and hidden states of partially observable systems. DTL can express properties involving uncertainty and likelihood that cannot be described by existing logics. A co-safe formulation of DTL is defined and algorithmic procedures are given for monitoring executions of a partially observable Markov decision process with respect to such formulae. A simulation case study of a rescue robotics application outlines our approach.