Sachin Adlakha

Mean Field Equilibrium in Dynamic Games with Strategic Complementarities

We study a class of stochastic dynamic games that exhibit strategic complementarities between players; formally, in the games we consider, the payoff of a player has increasing differences between her own state and the empirical distribution of the states of other players. Such games can be used to model a diverse set of applications, including network security models, recommender systems, and dynamic search in markets. Stochastic games are generally difficult to analyze, and these difficulties are only exacerbated when the number of players is large (as might be the case in the preceding examples).We consider an approximation methodology called mean field equilibrium to study these games. In such an equilibrium, each player reacts to only the long-run average state of other players. We find necessary conditions for the existence of a mean field equilibrium in such games. Furthermore, as a simple consequence of this existence theorem, we obtain several natural monotonicity properties. We show that there exist a “largest” and a “smallest” equilibrium among all those where the equilibrium strategy used by a player is nondecreasing, and we also show that players converge to each of these equilibria via natural myopic learning dynamics ; as we argue, these dynamics are more reasonable than the standard best-response dynamics. We also provide sensitivity results, where we quantify how the equilibria of such games move in response to changes in parameters of the game (for example, the introduction of incentives to players).

Joint Capacity, Flow and Rate Allocation for Multiuser Video Streaming Over Wireless Ad-Hoc Networks

Simultaneous support of multiple delay-critical application sessions such as multiuser video streaming require a paradigm shift in the design of ad-hoc wireless networks. Instead of the conventional layered approach, cross-layer optimization is needed for more efficient resource allocation, across the protocol stack and among multiple users. In this work, we extend our previous effort in joint capacity and flow assignment at the MAC and network layers, to include rate allocation at the application layer of each user. The proposed optimization aims to minimize the tradeoff between encoded video quality of all users versus overall network congestion. Compared to a scheme with oblivious layers, where capacity, flow and video rates are assigned individually, simulation results show significant performance gain of our proposed cross-layer approach, in terms of maximum sustainable rate and quality of the video streams.

Energy procurement strategies in the presence of intermittent sources

The increasing penetration of intermittent, unpredictable renewable energy sources such as wind energy, poses significant challenges for utility companies trying to incorporate renewable energy in their portfolio. In this work, we study the problem of conventional energy procurement in the presence of intermittent renewable resources. We model the problem as a variant of the newsvendor problem, in which the presence of renewable resources induces supply side uncertainty, and in which conventional energy may be procured in three stages to balance supply and demand. We compute closed-form expressions for the optimal energy procurement strategy and study the impact of increasing renewable penetration, and of proposed changes to the structure of electricity markets. We explicitly characterize the impact of a growing renewable penetration on the procurement policy by considering a scaling regime that models the aggregation of unpredictable renewable sources. A key insight from our results is that there is a separation between the impact of the stochastic nature of this aggregation, and the impact of market structure and forecast accuracy. Additionally, we study the impact on procurement of two proposed changes to the market structure: the addition and the placement of an intermediate market. We show that addition of an intermediate market does not necessarily increase the efficiency of utilization of renewable sources. Further, we show that the optimal placement of the intermediate market is insensitive to the level of renewable penetration.

Networked Markov Decision Processes With Delays

We consider a networked control system, where each subsystem evolves as a Markov decision process with some extra inputs from other systems. Each subsystem is coupled to its neighbors via communication links over which the signals are delayed, but are otherwise transmitted noise-free. A centralized controller receives delayed state information from each subsystem. The control action applied to each subsystem takes effect after a certain delay rather than immediately. We give an explicit bound on the finite history of measurement and control that is required for the optimal control of such networked Markov decision processes. We also show that these bounds depend only on the underlying graph structure as well as the associated delays. Thus, the partially observed Markov decision process associated with a networked Markov decision process can be converted into an information state Markov decision process, whose state does not grow with time.

Oblivious equilibrium: An approximation to large population dynamic games with concave utility

We study stochastic games with a large number of players, where players are coupled via their payoff functions. A standard solution concept for such games is Markov perfect equilibrium (MPE). It is well known that the computation of MPE suffers from the “curse of dimensionality.” Recently an approximate solution concept called “oblivious equilibrium” (OE) was developed by Weintraub et. al, where each player reacts to only the average behavior of other players. In this work, we characterize a set of games in which OE approximates MPE. Specifically, we show that if system dynamics and payoff functions are concave in state and action and have decreasing differences in state and action, then an oblivious equilibrium of such a game approximates MPE. These exogenous conditions on model primitives allow us to characterize a set of games where OE can be used as an approximate solution concept.

Optimal contract for wind power in day-ahead electricity markets

The growth of wind energy production poses several challenges in its integration in current electric power systems. In this work, we study how a wind power producer can bid optimally in existing electricity markets. We derive optimal contract size and expected profit for a wind producer under arbitrary penalty function and generation costs. A key feature of our analysis is to allow for the wind producer to strategically withhold production once the day ahead contract is signed. Such strategic behavior is detrimental to the smooth functioning of electricity markets. We show that under simple conditions on the offered price and marginal imbalance penalty, a risk neutral profit maximizing wind power producer will produce as much as wind power is available (up to its contract size).

On oblivious equilibrium in large population stochastic games

We study stochastic games with a large number of players, where players are coupled via their payoff functions. A standard solution concept for such games is Markov perfect equilibrium (MPE). It is well known that the computation of MPE suffers from the “curse of dimensionality.” To deal with this complexity, several researchers have introduced the idea of oblivious equilibrium (OE). In OE, each player reacts to only the long-run average state of other players. In this paper, we study existence of OE, and also find conditions under which OE approximates MPE well.

Mean field equilibrium in dynamic games with complementarities

We study stochastic dynamic games with a large number of players, where players are coupled via their payoff functions. We consider mean field equilibrium for such games: in such an equilibrium, each player reacts to only the long run average state of other players. In this paper we focus on a special class of stochastic games, where a player experiences strategic complementarities from other players; formally the payoff of a player has increasing differences between her own state and the aggregate empirical distribution of the states of other players. We find necessary conditions for the existence of a mean field equilibrium in such games. Furthermore, we show that there exist a “largest” and “smallest” equilibrium among all those where the equilibrium strategy used by a player is nondecreasing.

Optimal control of distributed Markov decision processes with network delays

We consider the problem of finding an optimal feedback controller for a network of interconnected subsystems, each of which is a Markov decision process. Each subsystem is coupled to its neighbors via communication links by which signals are delayed but are otherwise transmitted noise-free. One of the subsystems receives input from a controller, and the controller receives delayed state- measurements from all of the subsystems. We show that an optimal controller requires only a finite amount of memory which does not grow with time, and obtain a bound on the amount of memory that a controller needs to have for each subsystem. This makes the computation of an optimal controller through dynamic programming tractable. We illustrate our result by a numerical example, and show that it generalizes previous results on Markov decision processes with delayed state measurements.

Optimal Sensing Rate for Estimation over Shared Communication Links

In this paper we consider the problem of finding the optimal sensing rate for estimating an event over shared communication links. Since the communication link is shared between several systems sending their observations, not all observations can be sent to their respective estimators. To improve the performance of the estimator, we propose to oversample the event. In particular, we derive the minimum oversampling rate such that the maximum error covariance of the estimate is kept below the desired threshold. We derive a closed form equation for this minimum sampling rate that can be used in designing estimation techniques for systems with bandwidth constraints.