Ritchie Lee

Cyber-Physical Security: A Game Theory Model of Humans Interacting Over Control Systems

Recent years have seen increased interest in the design and deployment of smart grid devices and control algorithms. Each of these smart communicating devices represents a potential access point for an intruder spurring research into intruder prevention and detection. However, no security measures are complete, and intruding attackers will compromise smart grid devices leading to the attacker and the system operator interacting via the grid and its control systems. The outcome of these machine-mediated human-human interactions will depend on the design of the physical and control systems mediating the interactions. If these outcomes can be predicted via simulation, they can be used as a tool for designing attack-resilient grids and control systems. However, accurate predictions require good models of not just the physical and control systems, but also of the human decision making. In this manuscript, we present an approach to develop such tools, i.e., models of the decisions of the cyber-physical intruder who is attacking the systems and the system operator who is defending it, and demonstrate its usefulness for design.

Towards a bayesian network game framework for evaluating DDoS attacks and defense

With a long history of compromising Internet security, Distributed Denial-of-Service (DDoS) attacks have been intensively investigated and numerous countermeasures have been proposed to defend against them. In this work, we propose a non-standard game-theoretic framework that facilitates evaluation of DDoS attacks and defense. Our framework can be used to study diverse DDoS attack scenarios where multiple layers of protection are deployed and a number of uncertain factors affect the decision making of the players, and it also allows us to model different sophistication levels of reasoning by both the attacker and the defender. We conduct a variety of experiments to evaluate DDoS attack and defense scenarios where one or more layers of defense mechanisms are deployed, and demonstrate that our framework sheds light on the interplay between decision makings of both the attacker and the defender, as well as how they affect the outcomes of DDoS attack and defense games.

Game Theoretic Modeling of Pilot Behavior During Mid-Air Encounters

We show how to combine Bayes nets and game theory to predict the behavior of hybrid systems involving both humans and automated components. We call this novel framework “Semi Network-Form Games”, and illustrate it by predicting aircraft pilot behavior in potential near mid-air collisions. At present, at the beginning of such potential collisions, a collision avoidance system in the aircraft cockpit advises the pilots what to do to avoid the collision. However studies of mid-air encounters have found wide variability in pilot responses to avoidance system advisories. In particular, pilots rarely perfectly execute the recommended maneuvers, despite the fact that the collision avoidance system’s effectiveness relies on their doing so. Rather pilots decide their actions based on all information available to them (advisory, instrument readings, visual observations). We show how to build this aspect into a semi network-form game model of the encounter and then present computational simulations of the resultant model.

Neural network estimation of photovoltaic I–V curves under partially shaded conditions

In this paper, we present a neural network algorithm to estimate the I-V curve of a photovoltaic (PV) module under non-uniform temperature and shading distributions. We first present a novel photovoltaic simulation model which includes the interaction of (1) heat transfer including conduction, convection, and radiation (long and short wavelength), (2) an electro-optical two diode model including ohmic heat dissipation, and (3) environmental influences including shading, irradiance, and wind dependencies. The neural network trains on inputs which consist of shading and temperature patterns of each cell of the module, and predicts the current versus voltage and power versus voltage landscapes. This information can be used for maximum power point tracking under non-uniform conditions. The neural network was validated on the simulation model and on data collected from our rooftop PV lab.

Counter-Factual Reinforcement Learning: How to Model Decision-Makers That Anticipate the Future

This chapter introduces a novel framework for modeling interacting humans in a multi-stage game. This ”iterated semi network-form game” framework has the following desirable characteristics: (1) Bounded rational players, (2) strategic players (i.e., players account for one another’s reward functions when predicting one another’s behavior), and (3) computational tractability even on real-world systems. We achieve these benefits by combining concepts from game theory and reinforcement learning. To be precise, we extend the bounded rational ”level-K reasoning” model to apply to games over multiple stages. Our extension allows the decomposition of the overall modeling problem into a series of smaller ones, each of which can be solved by standard reinforcement learning algorithms. We call this hybrid approach ”level-K reinforcement learning”. We investigate these ideas in a cyber battle scenario over a smart power grid and discuss the relationship between the behavior predicted by our model and what one might expect of real human defenders and attackers.

An Integrated Safety and Systems Engineering Methodology for Small Unmanned Aircraft Systems

This paper presents an integrated methodology for addressing safety concerns during the systems engineering of small Unmanned Aircraft Systems (sUAS). We describe both the systems and safety engineering activities performed, their interrelations, and how they complement range safety analysis. The broad goal of this work is to support the derivation of an airworthiness statement, and the subsequent application for a Certificate of Authorization (COA) to operate sUAS in the National Airspace System (NAS). We exemplify our methodology by presenting its application to the Swift UAS and its payload data system, both of which are under development at NASA Ames Research Center.

Adaptive stress testing of airborne collision avoidance systems

This paper presents a scalable method to efficiently search for the most likely state trajectory leading to an event given only a simulator of a system. Our approach uses a reinforcement learning formulation and solves it using Monte Carlo Tree Search (MCTS). The approach places very few requirements on the underlying system, requiring only that the simulator provide some basic controls, the ability to evaluate certain conditions, and a mechanism to control the stochasticity in the system. Access to the system state is not required, allowing the method to support systems with hidden state. The method is applied to stress test a prototype aircraft collision avoidance system to identify trajectories that are likely to lead to near mid-air collisions. We present results for both single and multi-threat encounters and discuss their relevance. Compared with direct Monte Carlo search, this MCTS method performs significantly better both in finding events and in maximizing their likelihood.

Using Game Theoretic Models to Predict Pilot Behavior in NextGen Merging and Landing Scenario

This paper presents an implementation of the Semi Network-Form Game framework to predict pilot behavior in a merging and landing scenario. In this scenario, two aircraft are approaching to a freeze horizon with approximately equal distance when they become aware of each other via an Automatic Dependent Surveillance-Broadcast (ADS-B) communication link that will be available in Next Generation Air Transportation System (NextGen) airspace. Both pilots want to gain advantage over the other by entering the freeze horizon earlier and obtain the first place in landing. They re-adjust their speed accordingly. However, they cannot simply increase their speed to the maximum allowable values since they are concerned with safety, separation distance, effort, possibility of being vectored-off from landing and possibility of violating speed constraints. The authors present how to model these concerns and the rest of the system using semi network-form game framework. Using this framework, based on certain assumptions on pilot utility functions and on system configuration, estimates of pilot behavior and overall system evolution in time are provided. The possible employment of this modeling tool for airspace design optimization is also discussed. To support this discussion, a case is provided where the authors investigate the effect of increasing the merging point speed limit on the commanded speed distribution and on the percentage of vectored aircraft.

A Perception and Mapping Approach for Plume Detection in Payload Directed Flight

Payload Directed Flight (PDF) is a research task under NASA Aeronautics Research Mission Directorate, Fundamental Aeronautics Program, Subsonic Fixed Wing Project. In recent years, the capabilities, cardinality, and utilization of onboard sensor and payload suites have been greatly increasing throughout the aviation industry in various flight applications. Through intelligent coupling between the autopilot, onboard and payload sensors, next generation aircraft can take advantage of greater data availability to better achieve their mission objectives. Whether the objectives are mapping, tracking, or surveillance, the ability to close the loop around payload and non-traditional sensors using intelligent variably-autonomous algorithms enables increased efficiency and performance as well as exciting next-generation capabilities to vehicle platforms. Mapping and surveillance of dynamic and amorphous phenomena remains a difficult problem in the perception area of PDF research. As an example application, this research examines an algorithm for wildfire plume mapping by extending recent work in computer vision and robotic mapping. In a realistic simulation of a plume-tracking mission, results demonstrate that the vehicle was able to successfully map and track a dynamic plume in real-time to approximate visual accuracy.

Payload-Directed Control of Geophysical Magnetic Surveys

Using non-navigational (e.g. imagers, scientific) sensor information in control loops is a difficult problem to which no general solution exists. Whether the task can be successfully achieved in a particular case depends highly on problem specifics, such as application domain and sensors of interest. In this study, we investigate the feasibility of using magnetometer data for control feedback in the context of geophysical magnetic surveys. An experimental system was created and deployed to (a) assess sensor integration with autonomous vehicles, (b) investigate how magnetometer data can be used for feedback control, and (c) evaluate the feasibility of using such a system for geophysical magnetic surveys. Finally, we report the results of our experiments and show that payload-directed control of geophysical magnetic surveys is indeed feasible.