Setareh Maghsudi

Hybrid Centralized–Distributed Resource Allocation for Device-to-Device Communication Underlaying Cellular Networks

The basic idea of device-to-device (D2D) communication is that pairs of suitably selected wireless devices reuse the cellular spectrum to establish direct communication links, provided that the adverse effects of D2D communication on cellular users are minimized and that cellular users are given higher priority in using limited wireless resources. Despite its great potential in terms of coverage and capacity performance, implementing this new concept poses some challenges, particularly with respect to radio resource management. The main challenges arise from a strong need for distributed D2D solutions that operate in the absence of precise channel and network knowledge. To address this challenge, this paper studies a resource allocation problem in a single-cell wireless network with multiple D2D users sharing the available radio frequency channels with cellular users. We consider a realistic scenario where the base station (BS) is provided with strictly limited channel knowledge, whereas D2D and cellular users have no information. We prove a lower bound for the cellular aggregate utility in the downlink with fixed BS power, which allows for decoupling the channel allocation and D2D power control problems. An efficient graph-theoretical approach is proposed to perform channel allocation, which offers flexibility with respect to allocation criteria (aggregate utility maximization, fairness, and quality-of-service (QoS) guarantee). We model the power control problem as a multiagent learning game. We show that the game is an exact potential game with noisy rewards, which is defined on a discrete strategy set, and characterize the set of Nash equilibria. Q-learning better-reply dynamics is then used to achieve equilibrium.

Channel Selection for Network-Assisted D2D Communication via No-Regret Bandit Learning With Calibrated Forecasting

We consider the distributed channel selection problem in the context of device-to-device (D2D) communication as an underlay to a cellular network. Underlaid D2D users communicate directly by utilizing the cellular spectrum, but their decisions are not governed by any centralized controller. Selfish D2D users that compete for access to the resources form a distributed system where the transmission performance depends on channel availability and quality. This information, however, is difficult to acquire. Moreover, the adverse effects of D2D users on cellular transmissions should be minimized. In order to overcome these limitations, we propose a network-assisted distributed channel selection approach in which D2D users are only allowed to use vacant cellular channels. This scenario is modeled as a multi-player multi-armed bandit game with side information, for which a distributed algorithmic solution is proposed. The solution is a combination of no-regret learning and calibrated forecasting, and can be applied to a broad class of multi-player stochastic learning problems, in addition to the formulated channel selection problem. Theoretical analysis shows that the proposed approach not only yields vanishing regret in comparison to the global optimal solution but also guarantees that the empirical joint frequencies of the game converge to the set of correlated equilibria.

Distributed User Association in Energy Harvesting Dense Small Cell Networks: A Mean-Field Multi-Armed Bandit Approach

The emerging ultra-dense small cell networks (UD-SCNs) will need to combat a variety of challenges. On the one hand, massive number of devices sharing the limited wireless resources renders centralized control mechanisms infeasible due to the excessive cost of information acquisition and computation. On the other hand, to reduce the energy consumption from fixed power grid and/or battery, network entities (e.g., small cell base stations and user devices) may need to rely on the energy harvested from the ambient environment (e.g., from environmental sources). However, opportunistic energy harvesting introduces uncertainty in the network operation. In this paper, we study the distributed user association problem for energy harvesting UD-SCNs. After reviewing the state-of-the-art research, we outline the major challenges that arise in the presence of energy harvesting due to the uncertainty (e.g., limited knowledge on energy harvesting process or channel profile) as well as limited computational capacities. Finally, we propose an approach based on the mean-field multi-armed bandit games to solve the uplink user association problem for energy harvesting devices in a UD-SCN in the presence of uncertainty.

Joint channel allocation and power control for underlay D2D transmission

We study a joint channel allocation and power control problem for device-to-device (D2D) transmission underlaying a conventional single-cell cellular network. In such networks, direct transmissions are allowed among device pairs with local needs, provided that the adverse effects of D2D communications on cellular users is negligible and cellular users are given the priority in using limited wireless resources. Moreover, as D2D users are not in contact with the base station (BS), providing them with channel and/or network knowledge imposes excessive overhead. As a result, it becomes imperative to seek for new resource management mechanisms that fit the limitations of this concept. In this paper we consider a realistic model with respect to the information availability, and propose a joint channel allocation and power control scheme by using game- and graph theory. In particular, we first decompose the resource management problem into two cascaded channel allocation and power control problems, by proving a lower bound on the aggregate utility of cellular users. Afterwards we propose a centralized graph-theoretical channel allocation approach jointly for D2D and cellular users. Given the channel allocation, the subsequent power control problem is modeled as a game with incomplete information. We analyze the characteristics of this game and solve it in a distributed manner, by using a multi-agent Q-learning strategy. We evaluate the proposed resource allocation scheme both analytically and numerically.

Distributed User Association in Energy Harvesting Small Cell Networks: A Probabilistic Bandit Model

We investigate a distributed downlink user association problem in a dynamic small cell network, where every small base station (SBS) obtains its required energy through ambient energy harvesting. On the one hand, energy harvesting is inherently opportunistic, so that the amount of available energy is a random variable. On the other hand, users arrive at random and require different wireless services, rendering the energy consumption a random variable. In this paper, we develop a probabilistic framework to mathematically model and analyze the random behavior of energy harvesting and energy consumption. We further analyze the probability of QoS satisfaction (success probability), for each user with respect to every SBS. The proposed user association scheme is distributed in the sense that every user independently selects its corresponding SBS with the success probability serving as the performance metric. The success probability however depends on a variety of random factors such as energy harvesting, channel quality, and network traffic, whose distribution or statistical characteristics might not be known at users. Since acquiring the knowledge of these random variables (even statistical) is very costly in a dense network, we develop a bandit-theoretical formulation for distributed SBS selection when no prior information is available at users. The performance is analyzed both theoretically and numerically.

Game Theoretic Mechanisms for Resource Management in Massive Wireless IoT Systems

As a result of rapid advancement in communication technologies, the Internet of Things (i.e., ubiquitous connectivity among a very large number of persons and physical objects) is now becoming a reality. Nonetheless, a variety of challenges remain to be addressed, one of them being the efficient resource management in IoT. On one hand, central resource allocation is infeasible for large numbers of entities, due to excessive computational cost as well as immoderate overhead required for information acquisition. On the other hand, the devices connecting to IoT are expected to act smart, making decisions and performing tasks without human intervention. These characteristics render distributed resource management an essential feature of future IoT. Traditionally, game theory is applied to effectively analyze the interactive decision making of agents with conflicting interests. Nevertheless, conventional game models are not adequate to model large-scale systems, since they suffer from many shortcomings including analytical complexity, slow convergence, and excessive overhead due to information acquisition/exchange. In this article, we explore some non-conventional game theoretic models that fit the inherent characteristics of future large-scale IoT systems. Specifically, we discuss evolutionary games, mean field games, minority games, mean field bandit games, and mean field auctions. We provide the basics of each of these game models and discuss the potential IoT-related resource management problems that can be solved by using these models. We also discuss challenges, pitfalls, and future research directions.

Transmission mode selection for network-assisted device to device communication: A Levy-bandit approach

This paper studies device-to-device (D2D) communication underlaying cellular infrastructure, where each device pair is provided with two transmission modes: indirect and direct. Indirect transmission is a two-hop interference-free transmission via a base station. Despite being interference-free, this transmission type might be inefficient in communications scenarios where short-distance connections can be established. Moreover, the need for centralized resource allocation and utilizing extra hardware may lead to excessive complexity and unacceptable costs. In such scenarios, direct transmissions can utilize the proximity- and hop gains to achieve higher rates and lower end-to-end latencies. While having a potential for huge performance gains, direct D2D communications poses some fundamental challenges resulting from the absence of a devoted controller such as uncoordinated interference and unavailability of permanent direct channels. Roughly speaking, in an average sense, while indirect transmission pays safe and steady reward, direct transmission is risky, yielding a stochastic reward which might be lower than the guaranteed reward of indirect transmission, despite the proximity-and hop gains. Transmitters should therefore choose the most efficient transmission mode in the presence of limited information. This paper characterizes the reward process for each transmission mode to model the mode selection problem as a two-armed Levy-bandit game. Accordingly, the reward of the risky arm (direct mode) is considered to be a pure-jump Levy process, following compound Poisson distribution. Mathematical results from bandit and learning theories are used to solve the selection problem. Numerical results complete the paper.

Distributed User Association in Energy Harvesting Small Cell Networks: An Exchange Economy With Uncertainty

We consider a user association problem in the downlink of a small cell network, where small cells are powered solely through ambient energy harvesting. Since energy harvesting is opportunistic in nature, the amount of harvested energy is a random variable, without a priori known statistical characteristics. Thus, at the time of user association, the amount of available energy is unknown. We model the network as a competitive market with uncertainty, where self-interested small cells, modeled as consumers, are willing to maximize their utility scores by selecting users, represented as commodities. The utility scores of small cells depend on the random amount of harvested energy, formulated as nature’s state. For this model we prove the existence of Arrow-Debreu equilibrium. In addition, we analyze the equilibrium with respect to uniqueness, Pareto-efficiency, and social-optimality. Moreover, based on the Walras’ tatonnoment process and the static knapsack problem, we develop an efficient distributed user association scheme which converges to the equilibrium. We demonstrate the effectiveness of our proposed model through intensive numerical analysis.

Multi-armed bandit channel selection for power line communication

We consider a multi-channel power line communication (PLC) system, where channel coefficients follow non-identical log-normal distributions. We assume that statistical characteristics of each channel are time-variant, or in other words, channels are non-stationary. In such scenario, we formulate a channel selection problem, where a transmitter, provided with no prior information, aims at selecting the best channel among available PLC channels, so that the average utility, expressed in terms of data rate, is maximized. We cast the formulated channel selection problem as a piece-wise stationary multi-armed bandit game, and solve it by using algorithmic solutions. Numerical analysis establishes the applicability and effectiveness of our proposed model and solution.

Minority Games With Applications to Distributed Decision Making and Control in Wireless Networks

5G dense SCNs are expected to meet the thousand- fold mobile traffic challenge within the next few years. When developing solution schemes for resource allocation problems in such networks, conventional centralized control may no longer be viable due to excessive computational complexity and large signaling overhead caused by the large number of users and network nodes in such a network. Instead, distributed resource allocation (or decision making) methods with low complexity would be desirable to make the network self-organizing and autonomous. Minority game (MG) has recently gained the attention of the research community as a tool to model and solve distributed resource allocation problems. The main objective of this article is to study the applicability of MG to solve the distributed decision making problems in future wireless networks. We present the fundamental theoretical aspects of basic MG, some variants of MG, and the notion of equilibrium. We also study the current state of the art on the applications of MGs in communication networks. Furthermore, we describe an example application of MG to SCNs, where the problem of computation offloading by users in an SCN is modeled and analyzed using MG.