Roland P. Malhamé

Large-Population Cost-Coupled LQG Problems With Nonuniform Agents: Individual-Mass Behavior and Decentralized

We consider linear quadratic Gaussian (LQG) games in large population systems where the agents evolve according to nonuniform dynamics and are coupled via their individual costs. A state aggregation technique is developed to obtain a set of decentralized control laws for the individuals which possesses an epsiv-Nash equilibrium property. A stability property of the mass behavior is established, and the effect of inaccurate population statistics on an isolated agent is also analyzed by variational techniques.

\varepsilon

We consider uplink power control for lognormal fading channels in the large population case. First, we examine the structure of the control law in a centralized stochastic optimal control setup. We analyze the effect of large populations on the individual control inputs. Next, we split the centralized cost to approach the problem in a game theoretic framework. In this context, we introduce an auxiliary LQG control system and analyze the resulting /spl epsiv/-Nash equilibrium for the control law; subsequently we generalize the methodology developed for the LQG problem to the wireless power control problem to get an approximation for the collective effect of all other users on a given user. The obtained state aggregation technique leads to highly localized control configurations in contrast to the full state based optimal control strategy.

-Nash Equilibria

A statistical approach is used to model the dynamics of the electric demand of large aggregates of electric space heaters or air conditioners. The importance of such loads is twofold. First, they account for a significant portion of power system dynamics following a power outage. Second, because they are associated with energy storage, they are often selected for load shedding within a load management program. The derivation of the aggregate electrical dynamics is considered first for a homogeneous group of devices. Subsequently, a perturbation analysis yields the dynamics for a nonhomogeneous group. The homogeneons group aggregrate load model is a system of coupled ordinary, and partial differential equations (Fokker-Planck equations). It is obtained by writing evolution equations for the probability density of a hybrid-state Markov system used to model the switching dynamics of individual devices. This result is new and could give a clue to the analysis of a broad class of hybrid-state stochastic systems. In turn, this could provide a new impetus not only in the area of electric load modeling but other areas such as power system reliability and the design of relay control systems, where stochastic hybrid-state models occur frequently. Simulation results which illustrate the dynamical properties of the model are presented.

Individual and mass behaviour in large population stochastic wireless power control problems: centralized and Nash equilibrium solutions

We study a class of linear-quadratic-Gaussian (LQG) control problems with N decision makers, where the basic objective is to minimize a social cost as the sum of N individual costs containing mean field coupling. The exact socially optimal solution (determining a particular Pareto optimum) requires centralized information for each agent and has high implementational complexity. As an alternative we subsequently exploit a mean field structure in the centralized optimal control problem to develop decentralized cooperative optimization so that each agent only uses its own state and a function which may be computed offline; the resulting set of strategies asymptotically achieves the social optimum as N → ∞. A key feature in this scheme is to let each agent optimize a new cost as the sum of its own cost and another component capturing its social impact on all other agents. We also discuss the relationship between the decentralized cooperative solution and the so-called Nash Certainty Equivalence based solution presented in previous work on mean field LQG games.

Electric load model synthesis by diffusion approximation of a high-order hybrid-state stochastic system

The problem of load modeling for demand side management (DSM) purposes is addressed. The proposed load models rely on information about both the physical characteristics of elemental load devices at the distribution level and usage statistics of these devices. Although the class of models discussed has been previously proposed in the literature, its suitability for DSM purposes is definitely established by showing how the models can be a tool for real DSM actions evaluation. Some results are shown. >

Social Optima in Mean Field LQG Control: Centralized and Decentralized Strategies

This paper considers mobile to base station power control for lognormal fading channels in wireless communication systems within a centralized information stochastic optimal control framework. Under a bounded power rate of change constraint, the stochastic control problem and its associated Hamilton-Jacobi-Bellman (HJB) equation are analyzed by the viscosity solution method; then the degenerate HJB equation is perturbed to admit a classical solution and a suboptimal control law is designed based on the perturbed HJB equation. When a quadratic type cost is used without a bound constraint on the control, the value function is a classical solution to the degenerate HJB equation and the feedback control is affine in the system power. In addition, in this case we develop approximate, but highly scalable, solutions to the HJB equation in terms of a local polynomial expansion of the exact solution. When the channel parameters are not known a priori, one can obtain on-line estimates of the parameters and get adaptive versions of the control laws. In numerical experiments with both of the above cost functions, the following phenomenon is observed: whenever the users have different initial conditions, there is an initial convergence of the power levels to a common level and then subsequent approximately equal behavior which converges toward a stochastically varying optimum.

A class of models for load management application and evaluation revisited

We consider dynamic games in large population conditions where the agents evolve according to non-uniform dynamics and are weakly coupled via their dynamics and the individual costs. A state aggregation technique is developed to obtain a set of decentralized control laws for the individuals which possesses an F-Nash equilibrium property. An attraction property of the mass behaviour is established. The methodology and the results contained in this paper reveal novel behavioral properties of the relationship of any given individual with respect to the mass of individuals in large-scale noncooperative systems of weakly coupled agents.

Uplink power adjustment in wireless communication systems: a stochastic control analysis

We consider large population dynamic games and illuminate methodological connections with the theory of interacting particle systems. Combined with the large population modelling, a Nash certainty equivalence (NCE) methodology is introduced for specifying the localized strategy selection of a given agent within the Nash equilibrium setting. The NCE methodology closely parallels that found in the study of uncontrolled interacting particle systems within the framework of the McKean-Vlasov equation (1966): for both problems the solution is derived by focussing on a single generic individual at a microscopic level and analyzing its interaction with the ensemble of the other individuals of which it is itself, in a statistical sense, a representative

Nash Equilibria for Large-Population Linear Stochastic Systems of Weakly Coupled Agents

Electric water heaters have been the focus of several previous studies because of their pervasiveness in power systems and their consequent potential importance when considering conservation through more efficient design and operation of the heaters. Also, because such devices are associated with an energy storage capability, they are often considered within load management by direct device control programs. Finally, they tend to be responsible for persistent system load transients in a cold load pickup situation. Understanding of the above issues can be greatly enhanced with the availability of a computer model of aggregate electric water heating loads. A physically-based such model is presented and its dynamic properties are investigated via numerical simulation under various operating conditions and parameter configurations. The results are analyzed in the paper. >

Nash Certainty Equivalence in Large Population Stochastic Dynamic Games: Connections with the Physics of Interacting Particle Systems

We study large population stochastic dynamic games where each agent assigns individually determined coupling strengths (with possible spatial interpretation) to the states of other agents in its performance function. The mean field methodology yields a set of decentralized controls which generates an -Nash equilibrium for the population of size . A key feature of the mean field approximation (here with localized interactions) is that the resulting th individual agents control law depends on that agents state and the precomputable weighted average trajectory of the collection of all agents each applying a decentralized control law.