Klaus G. Becker

Simulation of cooperative control system tasks using hedonistic multi-agents

As control system complexity increases, scalability often becomes a limiting constraint. Distributed systems and multi-agent systems are useful approaches in designing complex systems, but communications for coordination are often a limiting factor as systems scale up. Colonies of social insects achieve synergistic results beneficial to the entire colony even though their individual behaviors can be described by simple hedonistic algorithms, and their available communications are very limited. Cooperative actions emerge from simple fixed action patterns in these insects. Complex control systems formed from a multitude of simpler agents or subsystems, with constrained and limited communications channels may also achieve emergent cooperation. Advantages of such systems are reduced communications complexity, and reduced complexity in any single element of the systems, as well as improved robustness.

Natural monopoly equilibria: Nash and von Stackelberg solutions

Abstract This paper provides a game-theoretic analysis of price-sustainable industry configurations in natural monopoly. We show that the sustainable configuration is the unique Nash equilibrium, and that it is also von Stackelberg equilibrium if any firm has to fulfill all demand at the quoted price. Furthermore, given this assumption, even configurations that are not price-sustainable according to the contestability theory are Nash equilibria.

Reinforcement Learning agents applied to a class of control system problems

Some control system problems are sufficiently complex that it is difficult to define all of the decision weighting up-front. Reinforcement Learning (RL) can be applied to “tune” system performance. The Centipede Game from Economic Game Theory is used to simulate a control system problem with competing subsystem goals. RL agents are applied to the Centipede Game, which pits agents against each other in a game with increasing payoffs if cooperation is developed between the players. The Centipede Game has been studied both in theory and empirically using human players by other researchers. Human players are more inclined to cooperate and achieve the longer-term payoff than economic game theory would predict. This paper focuses on an experimental study of cooperation between RL players (agents) without explicit communications between the players. This work has application in control systems problems where communications is constrained.