Arman C. Kizilkale

Mean Field Stochastic Adaptive Control

For noncooperative games the mean field (MF) methodology provides decentralized strategies which yield Nash equilibria for large population systems in the asymptotic limit of an infinite (mass) population. The MF control laws use only the local information of each agent on its own state and own dynamical parameters, while the mass effect is calculated offline using the distribution function of i) the populations dynamical parameters, and ii) the populations cost function parameters, for the infinite population case. These laws yield approximate equilibria when applied in the finite population. In this paper, these a priori information conditions are relaxed, and incrementally the cases are considered where, first, the agents estimate their own dynamical parameters, and, second, estimate the distribution parameter in i) and ii) above. An MF stochastic adaptive control (SAC) law in which each agent observes a random subset of the population of agents is specified, where the ratio of the cardinality of the observed set to that of the number of agents decays to zero as the population size tends to infinity. Each agent estimates its own dynamical parameters via the recursive weighted least squares (RWLS) algorithm and the distribution of the populations dynamical parameters via maximum likelihood estimation (MLE). Under reasonable conditions on the population dynamical parameter distribution, the MF-SAC Law applied by each agent results in i) the strong consistency of the self parameter estimates and the strong consistency of the population distribution function parameters; ii) the long run average L2 stability of all agent systems; iii) a (strong)ϵ-Nash equilibrium for the population of agents for all ϵ > 0; and iv) the a.s. equality of the long run average cost and the non-adaptive cost in the population limit.

Large scale real-time bidding in the smart grid: A mean field framework

We model the power market as a dynamic large population game where suppliers and consumers submit their bids in real-time. The agents are coupled in their dynamics and cost functions through the price process. The control action computation complexity and information exchange requirements for each agent increase as the number of agents in the system increases, and this naturally leads to computational intractability. We apply the mean field methodology to study the limit behaviour of a large population of agents, and present a decentralized algorithm where agents submit their bids solely following the price signal and using statistical information that is measured from the entire population. We show that under some restrictions on the population parameter distributions the proposed algorithm gives rise to a situation where (i) all agent systems are L2 stable, and (ii) the set of controls yields an ε-Nash equilibrium.

Regulation and efficiency in markets with friction

We analyse the efficiency of markets with friction, particularly power markets. We model the market as a dynamic system with {d<inf>t</inf>; t ≥ 0} the demand process and {s<inf>t</inf>; t ≥ 0} the supply process. Using stochastic differential equations to model the dynamics with friction, we investigate the efficiency of the market under an integrated expected cost function. Under this model, an efficiency-volatility tradeoff theorem is presented.

Stochastic adaptive Nash Certainty Equivalence control: Population parameter distribution estimation

For noncooperative games the Nash Certainty Equivalence (NCE), or Mean Field (MF) methodology developed in previous work provides decentralized strategies which asymptotically yield Nash equilibria. The NCE (MF) control laws use only the local information of each agent on its own state evolution and knowledge of its own dynamical parameters, while the behaviour of the mass is precomputable from knowledge of the distribution of dynamical parameters throughout the mass population.

Mean Field (NCE) stochastic control: Populations of major and egoist-altruist agents

For noncooperative games the Nash Certainty Equivalence (NCE), or Mean Field (MF) methodology [1], [2] provides decentralized strategies which asymptotically yield Nash equilibria. An extension of this theory to populations of altruistic agents (defined with so-called social cost functions) and to mixed populations was carried out in [3], and a theory treating populations of egoistic agents and one or more so-called major agents was developed in [4]. In this paper we study the equilibria and the overall stability of dynamic LQG games, where (i) there is a single major agent and a large population of mixed minor agents, and (ii) the cost for each minor agent is a convex combination of its own cost and the social cost of the minor agents. We analyse the resulting equilibria, provide experimental results, and present a mean field stochastic control algorithm, which when applied by all agents in the system, gives rise to system behaviour where (i) all agents systems are L2 stable, (ii) the set of controls yields an ε-Nash equilibrium for all ε, and (iii) if each minor agent in the system only considers the social cost, then the difference between (i) the cost observed by each minor agent and (ii) the social cost that would be observed if a centralized controller minimizes the social cost tends to zero as the population size grows to infinity.

Volatility and efficiency in markets with friction

We consider a game theoretic model where multiple suppliers and consumers interact continuously by setting prices in a dynamic market with friction. Using stochastic differential equations to model the dynamics with friction, we investigate the equilibrium, and analyze the efficiency of the market under an integrated expected cost function. We provide an intriguing efficiency-volatility tradeoff theorem.

Regulation and double price mechanisms in markets with friction

In previous work we modeled the real-time power market as a dynamic system and presented an “efficiency-volatility” trade-off theorem stating that in markets with supply friction, an efficient market must have volatile prices. In this paper we introduce a novel market mechanism for power markets where there are two prices: one for the real-time power market for suppliers who have friction and another for frictionless ancillary supply with a marginal cost higher than that of regular suppliers. We show that for a given level of acceptable price volatility the double price system with the ancillary supplier is more efficient than the single price system without the frictionless ancillary supplier.

Emergence of coalitions in mean field stochastic systems

In this paper we define and investigate coalition formation for the large population dynamic LQG tracking game that was studied in [1], [2]. First, we present a mean field stochastic control algorithm which, when applied by all agents in the system, gives rise to system behaviour where (i) all agents systems are L2 stable, and (ii) the set of controls yields an ε-Nash equilibrium. Then, as the population size tends to infinity, we show that the system with a coalition state converges to a major-minor agent system studied in [3], where in the limit the coalition state acts as a single major agent. We investigate the existence properties of the coalition state, and further discuss the stability of the coalitions in the overall dynamic system with respect to the fraction of the agents in the coalition.

Duality of ancillary services and intermittent suppliers

We model the continuous-time power market as an optimal control problem where the demand and supply processes are governed by stochastic differential equations controlled by the price. We first analyze the efficiency of an intermittent supplier which is characterized by cheap production with high supply volatility. We show that for a sufficiently high intermittent supply volatility, the intermittent supplier negatively impacts the social efficiency. Next, we introduce a novel market mechanism for power markets: we define one price process for supply subject to friction and another price for frictionless ancillary supply with a marginal cost of production higher than that of the regular supply. We show that (i) the negative efficiency impact of the intermittent supplier due to stochasticity and uncontrollability can be offset with the new double price market mechanism, and (ii) the volatility of the price can be decreased with the new double price mechanism.