Hamidou Tembine

Distributed Strategic Learning for Wireless Engineers

Establishing the link between several theories, this book demonstrates what is needed to learn strategic interaction in wireless networks under uncertainty, randomness, and time delays. This robust game theory cookbook analyzes games where some parameters are uncertain and random. Starting with the fundamentals of strategic learning and game theory, the text then covers partially and fully distributed learning, combined learning for negotiation, bargaining solutions, Nash equilibria, Stackelberg solutions, correlated equilibria, conjectural variations, and satisfactory solutions. It concludes with practical algorithms and techniques to implement game theory in wireless networks.

Evolutionary Games in Wireless Networks

We consider a noncooperative interaction among a large population of mobiles that interfere with each other through many local interactions. The first objective of this paper is to extend the evolutionary game framework to allow an arbitrary number of mobiles that are involved in a local interaction. We allow for interactions between mobiles that are not necessarily reciprocal. We study 1) multiple-access control in a slotted Aloha-based wireless network and 2) power control in wideband code-division multiple-access wireless networks. We define and characterize the equilibrium (called evolutionarily stable strategy) for these games and study the influence of wireless channels and pricing on the evolution of dynamics and the equilibrium.

Mean field asymptotics of Markov Decision Evolutionary Games and teams

We introduce Markov Decision Evolutionary Games with N players, in which each individual in a large population interacts with other randomly selected players. The states and actions of each player in an interaction together determine the instantaneous payoff for all involved players. They also determine the transition probabilities to move to the next state. Each individual wishes to maximize the total expected discounted payoff over an infinite horizon. We provide a rigorous derivation of the asymptotic behavior of this system as the size of the population grows to infinity. We show that under any Markov strategy, the random process consisting of one specific player and the remaining population converges weakly to a jump process driven by the solution of a system of differential equations. We characterize the solutions to the team and to the game problems at the limit of infinite population and use these to construct almost optimal strategies for the case of a finite, but large, number of players. We show that the large population asymptotic of the microscopic model is equivalent to a (macroscopic) Markov decision evolutionary game in which a local interaction is described by a single player against a population profile. We illustrate our model to derive the equations for a dynamic evolutionary Hawk and Dove game with energy level.

Risk-Sensitive Mean-Field Games

In this paper, we study a class of risk-sensitive mean-field stochastic differential games. We show that under appropriate regularity conditions, the mean-field value of the stochastic differential game with exponentiated integral cost functional coincides with the value function satisfying a Hamilton -Jacobi- Bellman (HJB) equation with an additional quadratic term. We provide an explicit solution of the mean-field best response when the instantaneous cost functions are log-quadratic and the state dynamics are affine in the control. An equivalent mean-field risk-neutral problem is formulated and the corresponding mean-field equilibria are characterized in terms of backward-forward macroscopic McKean-Vlasov equations, Fokker-Planck-Kolmogorov equations, and HJB equations. We provide numerical examples on the mean field behavior to illustrate both linear and McKean-Vlasov dynamics.

The evolution of transport protocols: An evolutionary game perspective

Todays Internet is well adapted to the evolution of protocols at various network layers. Much of the intelligence of congestion control is delegated to the end users and they have a large amount of freedom in the choice of the protocols they use. In the absence of a centralized policy for a global deployment of a unique protocol to perform a given task, the Internet experiences a competitive evolution between various versions of protocols. The evolution manifests itself through the upgrading of existing protocols, abandonment of some protocols and appearance of new ones. We highlight in this paper the modeling capabilities of the evolutionary game paradigm for explaining past evolution and predicting the future one. In particular, using this paradigm we derive conditions under which (i) a successful protocol would dominate and wipe away other protocols, or (ii) various competing protocols could coexist. In the latter case we also predict the share of users that would use each of the protocols. We further use evolutionary games to propose guidelines for upgrading protocols in order to achieve desirable stability behavior of the system.

Dynamic Robust Games in MIMO Systems

In this paper, we study dynamic robust power-allocation games in multiple-input-multiple-output systems under the imperfectness of the channel-state information at the transmitters. Using a robust pseudopotential-game approach, we show the existence of robust solutions in both discrete and continuous action spaces under suitable conditions. Considering the imperfectness in terms of the payoff measurement at the transmitters, we propose a COmbined fully DIstributed Payoff and Strategy Reinforcement Learning (CODIPAS-RL) in which each transmitter learns its payoff function, as well as the associated optimal covariance matrix strategies. Under the heterogeneous CODIPAS-RL, the transmitters can use different learning patterns (heterogeneous learning) and different learning rates. We provide sufficient conditions for the almost-sure convergence of the heterogeneous learning to ordinary differential equations. Extensions of the CODIPAS-RL to Itôs stochastic differential equations are discussed.

Bio-inspired delayed evolutionary game dynamics with networking applications

We study in this paper some evolutionary games where competition between individuals from a large population occurs through many local interactions between randomly selected individuals. We focus on games that have the property of possessing a single interior evolutionarily stable strategy (ESS). We study in particular the effect of the time delays on the convergence of evolutionary dynamics to the ESS in an evolutionary game in which each pure strategy is associated with its own delay. In particular, we study a multiple access game as well as a Hawk and Dove game. We study the properties of the ESS in these games and also the effect of time delays on the convergence of various bio-inspired evolutionary game dynamics to the ESS.

Optimum and Equilibrium in Assignment Problems With Congestion: Mobile Terminals Association to Base Stations

The classic optimal transportation problem consists in finding the most cost-effective way of moving masses from one set of locations to another, minimizing its transportation cost. The formulation of this problem and its solution have been useful to understand various mathematical, economical, and control theory phenomena, such as, e.g., Witsenhausens counterexample in stochastic control theory, the principal-agent problem in microeconomic theory, location and planning problems, etc. In this work, we incorporate the effect of network congestion to the optimal transportation problem and we are able to find a closed form expression for its solution. As an application of our work, we focus on the mobile association problem in cellular networks (the determination of the cells corresponding to each base station). In the continuum setting, this problem corresponds to the determination of the locations at which mobile terminals prefer to connect (by also considering the congestion they generate) to a given base station rather than to other base stations. Two types of problems have been addressed: a global optimization problem for minimizing the total power needed by the mobile terminals over the whole network (global optimum), and a user optimization problem, in which each mobile terminal chooses to which base station to connect in order to minimize its own cost (user equilibrium). This work combines optimal transportation with strategic decision making to characterize both solutions.

Evolutionary game dynamics with migration for hybrid power control in wireless communications

We propose an evolutionary game dynamics with migration for hybrid population games with many local interactions at the same time. Each local interaction concerns a random number of interacting players. The strategies of a player have two components. Specifically, each player chooses both (i) the region or subpopulation and (ii) an action among a finite set of secondary pure strategies in each region. We assume that when updating a strategy, a player can change only the secondary strategies associate to the region at a time. We investigate what impact this restriction has on the population dynamics. We apply this model to the integrated power control and base station assignment problem in a multi-cell in code division multiple access (CDMA) wireless data networks with large number of mobiles. We show that global neutrally evolutionary stable strategies are stationary points of hybrid mean dynamics called dynamics with multicomponent strategies under the positive correlation conditions. We give some convergence results of our hybrid model in stable population games and potential population games.

Energy-constrained Mean Field Games in Wireless Networks

This article is part of a Special Issue on ICT-based Strategies for Environmental Conflicts. In this paper, we study anti-jamming problems in energy-aware wireless networks using mean field stochastic games. We consider three types of users: jammers, primary users and secondary users. When active, each secondary transmitter–receiver uses carrier sensing and is subject to a long-term energy constraint. We formulate the interaction between primary and large number of secondary users as a hierarchical mean field game. The proposed mean field framework allows one to describe the evolution of the remaining energy distribution and the location of the secondary users. We provide explicit optimal strategies for both primary and secondary users based on attackers strategies. Each secondary user reacts to the aggregative behavior of the others and manages its battery based on the anticipated complete characterization of the optimal distribution of energy.