James W. Bono

Cyber-Physical Security: A Game Theory Model of Humans Interacting Over Control Systems

Recent years have seen increased interest in the design and deployment of smart grid devices and control algorithms. Each of these smart communicating devices represents a potential access point for an intruder spurring research into intruder prevention and detection. However, no security measures are complete, and intruding attackers will compromise smart grid devices leading to the attacker and the system operator interacting via the grid and its control systems. The outcome of these machine-mediated human-human interactions will depend on the design of the physical and control systems mediating the interactions. If these outcomes can be predicted via simulation, they can be used as a tool for designing attack-resilient grids and control systems. However, accurate predictions require good models of not just the physical and control systems, but also of the human decision making. In this manuscript, we present an approach to develop such tools, i.e., models of the decisions of the cyber-physical intruder who is attacking the systems and the system operator who is defending it, and demonstrate its usefulness for design.

Counter-Factual Reinforcement Learning: How to Model Decision-Makers That Anticipate the Future

This chapter introduces a novel framework for modeling interacting humans in a multi-stage game. This ”iterated semi network-form game” framework has the following desirable characteristics: (1) Bounded rational players, (2) strategic players (i.e., players account for one another’s reward functions when predicting one another’s behavior), and (3) computational tractability even on real-world systems. We achieve these benefits by combining concepts from game theory and reinforcement learning. To be precise, we extend the bounded rational ”level-K reasoning” model to apply to games over multiple stages. Our extension allows the decomposition of the overall modeling problem into a series of smaller ones, each of which can be solved by standard reinforcement learning algorithms. We call this hybrid approach ”level-K reinforcement learning”. We investigate these ideas in a cyber battle scenario over a smart power grid and discuss the relationship between the behavior predicted by our model and what one might expect of real human defenders and attackers.

Decision-Theoretic Prediction and Policy Design of GDP Slot Auctions

We examine the potential for a simple auction to allocate arrival slots during Ground Delay Programs (GDP’s) more efficiently than the currently used sys- tem. The analysis of these auctions uses Predictive Game Theory (PGT) Wolpert and Bono (2010a,b), a new approach that produces a probability distribution over strategies instead of an equilibrium set. We compare the simple auction with other allocation methods, including combinatorial auctions and theoretical benchmarks using data from a one-hour GDP at Chicago Midway. We find that the simple slot auction overcomes several practical shortcomings of other approaches, while offering economically significant efficiency gains with respect to current practices and the potential to lower airline costs. We also find that the second price version of the simple auction dominates the first price version in nearly every decision-relevant category. This is despite the fact that none of the conventional arguments for second price auctions, such as dominant strategy implementability, even apply to GDP slot auctions. Finally, the results indicate that combinatorial auctions, if made operationally practical, might be more efficient than our auction, even though the combinatorial auction does not implement the social optimum in dominant strategies.

Predicting behavior in unstructured bargaining with a probability distribution

In experimental tests of human behavior in unstructured bargaining games, typically many joint utility outcomes are found to occur, not just one. This suggests we predict the outcome of such a game as a probability distribution. This is in contrast to what is conventionally done (e.g, in the Nash bargaining solution), which is predict a single outcome. We show how to translate Nashs bargaining axioms to provide a distribution over outcomes rather than a single outcome. We then prove that a subset of those axioms forces the distribution over utility outcomes to be a power-law distribution. Unlike Nashs original result, our result holds even if the feasible set is finite. When the feasible set is convex and comprehensive, the mode of the power law distribution is the Harsanyi bargaining solution, and if we require symmetry it is the Nash bargaining solution. However, in general these modes of the joint utility distribution are not the experimentalists Bayesoptimal predictions for the joint utility. Nor are the bargains corresponding to the modes of those joint utility distributions the modes of the distribution over bargains in general, since more than one bargain may result in the same joint utility. After introducing distributional bargaining solution concepts, we show how an external regulator can use them to optimally design an unstructured bargaining scenario. Throughout we demonstrate our analysis in computational experiments involving flight rerouting negotiations in the National Airspace System. We emphasize that while our results are formulated for unstructured bargaining, they can also be used to make predictions for noncooperative games where the modeler knows the utility functions of the players over possible outcomes of the game, but does not know the move spaces the players use to determine those outcomes.

Game Mining: How to Make Money from those about to Play a Game

It is known that a player in a noncooperative game can benefit by publicly re- stricting their possible moves before start of play. We show that, more generally, a player may benefit by publicly committing to pay an external party an amount that is contingent on the games outcome. We explore what happens when external parties (who we call game miners) discover this fact and seek to profit from it by entering an outcome-contingent contract with the players. We analyze various bargaining games between miners and players for determining such an outcome- contingent contract. We establish restrictions on the strategic settings in which a game miner can profit, and bounds on the game miners profit given various structured bargaining games. These bargaining games include playing the players against one another, as well as allowing the players to pay the miner(s) for exclu- sivity and first-mover advantage. We also establish that when all players can enter contracts with miners, to guarantee the existence of equilibria it is necessary to assume that players can randomize over the contracts they make.