Archie C. Chapman

Review: a unifying framework for iterative approximate best-response algorithms for distributed constraint optimization problems1

Distributed constraint optimization problems (DCOPs) are important in many areas of computer science and optimization. In a DCOP, each variable is controlled by one of many autonomous agents, who together have the joint goal of maximizing a global objective function. A wide variety of techniques have been explored to solve such problems, and here we focus on one of the main families, namely iterative approximate best-response algorithms used as local search algorithms for DCOPs. We define these algorithms as those in which, at each iteration, agents communicate only the states of the variables under their control to their neighbours on the constraint graph, and that reason about their next state based on the messages received from their neighbours. These algorithms include the distributed stochastic algorithm and stochastic coordination algorithms, the maximum-gain messaging algorithms, the families of fictitious play and adaptive play algorithms, and algorithms that use regret-based heuristics. This family of algorithms is commonly employed in real-world systems, as they can be used in domains where communication is difficult or costly, where it is appropriate to trade timeliness off against optimality, or where hardware limitations render complete or more computationally intensive algorithms unusable. However, until now, no overarching framework has existed for analyzing this broad family of algorithms, resulting in similar and overlapping work being published independently in several different literatures. The main contribution of this paper, then, is the development of a unified analytical framework for studying such algorithms. This framework is built on our insight that when formulated as non-cooperative games, DCOPs form a subset of the class of potential games. This result allows us to prove convergence properties of iterative approximate best-response algorithms developed in the computer science literature using game-theoretic methods (which also shows that such algorithms can also be applied to the more general problem of finding Nash equilibria in potential games), and, conversely, also allows us to show that many game-theoretic algorithms can be used to solve DCOPs. By so doing, our framework can assist system designers by making the pros and cons of, and the synergies between, the various iterative approximate best-response DCOP algorithm components clear.

A healthy dose of reality for game-theoretic approaches to residential demand response

This paper addresses the assumptions underpinning many control schemes for residential demand response (RDR), with particular focus on those that adopt the framework of non - cooperative games. We propose four principal assumptions that we believe are necessary to give a realistic grounding to research on RDR, so that they might be more readily applied to the real problems faced by aggregators and households in a future energy network. These are that: (i) The energy use levels of households do not take continuous values, they take discrete and hybrid values; (ii) In addition to the system state variables, each household has a private state, representing the states of the goals it addresses in consuming electricity; (iii) Households have private preferences that are state-based, and therefore non-convex and combinatorial, and moreover, the monetary costs imposed by the system operator represents only part of their preferences for electrical energy use; and (iv) Household behaviour is strategic, both at the level of equilibrium analysis and algorithmic design. For each assumption we argue why it is necessary that a RDR scheme satisfy it, and illustrate the effects of violating our proposed assumption, with reference the existing literature on RDR schemes. We also provide several examples of techniques that satisfy each assumption, and illustrate our assumptions by developing a model that satisfies all four.

CONVERGENT LEARNING ALGORITHMS FOR UNKNOWN REWARD GAMES

In this paper, we address the problem of convergence to Nash equilibria in games with rewards that are initially unknown and must be estimated over time from noisy observations. These games arise in many real-world applications, whenever rewards for actions cannot be prespecified and must be learned online, but standard results in game theory do not consider such settings. For this problem, we derive a multiagent version of

Decentralized Dynamic Task Allocation Using Overlapping Potential Games

\mathcal{Q}

A Fast Distributed Algorithm for Large-Scale Demand Response Aggregation

-learning to estimate the reward functions using novel forms of the

Benchmarking hybrid algorithms for distributed constraint optimisation games

\epsilon

Flood disaster mitigation: a real-world challenge problem for multi-agent unmanned surface vehicles

-greedy learning policy. Using these

A Faithful Distributed Mechanism for Demand Response Aggregation

\mathcal{Q}

An Iterative On-Line Auction Mechanism for Aggregated Demand-Side Participation

-learning schemes to estimate reward functions, we then provide conditions guaranteeing the convergence of adaptive play and the better-reply processes to Nash equilibria in potential games and games with more general forms of acyclicity, and of regret matching to the set of correlated equilibria in generic games. A secondary result is that we prove the strong ergoditicity of stochastic adaptive play and stochastic better-reply processes in the ca...

Evaluation of a multi-stage stochastic optimisation framework for energy management of residential PV-storage systems

This paper reports on a novel decentralized technique for planning agent schedules in dynamic task allocation problems. Specifically, we use a stochastic game formulation of these problems in which tasks have varying hard deadlines and processing requirements. We then introduce a new technique for approximating this game using a series of static potential games, before detailing a decentralized method for solving the approximating games that uses the distributed stochastic algorithm. Finally, we discuss an implementation of our approach to a task allocation problem in the RoboCup Rescue disaster management simulator. The results show that our technique performs comparably to a centralized task scheduler (within 6% on average), and also, unlike its centralized counterpart, it is robust to restrictions on the agents’ communication and observation ranges.