Fangwen Fu

Learning to Compete for Resources in Wireless Stochastic Games

In this paper, we model the various users in a wireless network (e.g., cognitive radio network) as a collection of selfish autonomous agents that strategically interact to acquire dynamically available spectrum opportunities. Our main focus is on developing solutions for wireless users to successfully compete with each other for the limited and time-varying spectrum opportunities, given experienced dynamics in the wireless network. To analyze the interactions among users given the environment disturbance, we propose a stochastic game framework for modeling how the competition among users for spectrum opportunities evolves over time. At each stage of the stochastic game, a central spectrum moderator (CSM) auctions the available resources, and the users strategically bid for the required resources. The joint bid actions affect the resource allocation and, hence, the rewards and future strategies of all users. Based on the observed resource allocations and corresponding rewards, we propose a best-response learning algorithm that can be deployed by wireless users to improve their bidding policy at each stage. The simulation results show that by deploying the proposed best-response learning algorithm, the wireless users can significantly improve their own bidding strategies and, hence, their performance in terms of both the application quality and the incurred cost for the used resources.

Detection of Spectral Resources in Cognitive Radios Using Reinforcement Learning

Available spectrum for wireless communications is a limited resource which gains importance with the increasing demand for mobile communication services with high data rates. Measurements show that assigned frequency bands (FBs) are not used efficiently. One approach for increasing the efficiency in spectrum use is the concept of overlay systems, which can be seen as an enabling technology for cognitive radio and dynamic spectrum access by providing frequency agility. In this paper, we propose an approach for the detection of spectral resources based on reinforcement learning, allowing the cognitive radio to select the FBs with the most available resources.

A New Systematic Framework for Autonomous Cross-Layer Optimization

Cross-layer optimization solutions have been proposed in recent years to improve the performance of wireless users that operate in a time-varying, error-prone network environment. However, these solutions often rely on centralized cross-layer optimization solutions that violate the layered network architecture of the protocol stack by requiring layers to provide access to their internal protocol parameters to other layers. This paper presents a new systematic framework for cross-layer optimization, which allows each layer to make autonomous decisions to maximize the wireless users utility by optimally determining what information should be exchanged among layers. Hence, this cross-layer framework preserves the current layered network architecture. Since the user interacts with the wireless environment at various layers of the protocol stack, the cross-layer optimization problem is solved in a layered fashion such that each layer adapts its own protocol parameters and exchanges information (messages) with other layers that cooperatively maximize the performance of the wireless user. Based on the proposed layered framework, we also design a message-exchange mechanism that determines the optimal cross-layer transmission strategies, given the users experienced environment dynamics.

Wireless Network Virtualization as A Sequential Auction Game

We propose a virtualization framework to separate the network operator (NO) who focuses on wireless resource management and service providers (SP) who target distinct objectives with different constraints. Within the proposed framework, we model the interactions among SPs and NO as a stochastic game, each stage of which is played by SPs (on behalf of the end users) and is regulated by the NO through the Vickrey-Clarke-Groves (VCG) mechanism. Due to the strong coupling between the future decisions of SPs and lack of global information at each SP, the stochastic game is notoriously hard. Instead, we introduce conjectural prices to represent the future congestion levels the end users potentially will experience, via which the future interactions between SPs are decoupled. Then, the policy to play the dynamic rate allocation game becomes selecting the conjectural prices and announcing a strategic value function (i.e., the preference on the rate) at each time. We prove that there exists one Nash equilibrium in the conjectural prices and, given the conjectural prices, the SPs have to truthfully reveal their own value function. We further prove that this Nash equilibrium results in efficient rate allocation in our virtualized wireless network. In other words, there are enough incentives for NO to advertise such a conjectural price and SPs to follow this advice.

Structure-Aware Stochastic Control for Transmission Scheduling

In this paper, we consider the problem of real-time transmission scheduling over time-varying channels. We first formulate this transmission scheduling problem as a Markov decision process and systematically unravel the structural properties (e.g., concavity in the state-value function and monotonicity in the optimal scheduling policy) exhibited by the optimal solutions. We then propose an online learning algorithm that preserves these structural properties and achieves ε-optimal solutions for an arbitrarily small ε. The advantages of the proposed online method are given as follows: 1) It does not require a priori knowledge of the traffic arrival and channel statistics, and 2) it adaptively approximates the state-value functions using piecewise linear functions and has low storage and computation complexity. We also extend the proposed low-complexity online learning solution to enable prioritized data transmission. The simulation results demonstrate that the proposed method achieves significantly better utility (or delay)-energy tradeoffs compared to existing state-of-the-art online optimization methods.

A Pricing Mechanism for Resource Allocation in Wireless Multimedia Applications

We consider the problem of multiuser resource allocation for wireless multimedia applications deployed by autonomous and noncollaborative wireless stations (WSTAs). Existing resource allocation solutions for WLANs are not network-aware and do not take into account the selfish behavior of individual WSTAs. Specifically, the selfish WSTAs can manipulate the network by untruthfully representing their private information (i.e., video characteristics, experienced channel conditions, and deployed streaming strategies). This often results in inefficient resource allocations. To overcome this obstacle, we present a pricing mechanism for message exchanges between the WSTAs and the Central Spectrum Moderator (CSM). The messages represent network-aware resource demands and corresponding prices. We prove that the message exchanges reach the Nash equilibrium and that the resulting equilibrium messages generate allocations which are efficient, budget balanced, and satisfy voluntary participation. The simulation results verify that these properties hold when the WSTAs behave strategically. Additionally, we evaluate the impact of initial prices and network congestion level on the convergence rate of message exchanges.

Fast intra prediction algorithm in H.264-AVC

Spatial intra prediction is introduced to achieve higher coding efficiency in H.264/AVC. Unfortunately, this comes at the cost in considerably increased complexity when RDO is enabled. In this paper, we propose a fast intra prediction algorithm including reducing the redundancy between the chroma mode decision and luma mode decision, improving the mode decision scheme and heuristically selecting block type. Experimental results show that the proposed algorithm can drastically reduce coding complexity with only negligible coding efficiency loss.

Decomposition Principles and Online Learning in Cross-Layer Optimization for Delay-Sensitive Applications

In this paper, we propose a general cross-layer optimization framework for delay-sensitive applications over single wireless links in which we explicitly consider both the heterogeneous and dynamically changing characteristics (e.g., delay deadlines, dependencies, distortion impacts, etc.) of delay-sensitive applications and the underlying time-varying channel conditions. We first formulate this problem as a nonlinear constrained optimization by assuming complete knowledge of the application characteristics and the underlying channel conditions. This constrained cross-layer optimization is then decomposed into several subproblems, each corresponding to the cross-layer optimization for one DU. The proposed decomposition method explicitly considers how the cross-layer strategies selected for one DU will impact its neighboring DUs as well as the DUs that depend on it through the resource price (associated with the resource constraint) and neighboring impact factors (associated with the scheduling constraints). However, the attributes (e.g., distortion impact, delay deadline, etc.) of future DUs as well as the channel conditions are often unknown in the considered real-time applications. In this case, the cross-layer optimization is formulated as a constrained Markov decision process (MDP) in which the impact of current cross-layer actions on the future DUs can be characterized by a state-value function. We then develop a low-complexity cross-layer optimization algorithm using online learning for each DU transmission. This online optimization utilizes information only about the previous transmitted DUs and past experienced channel conditions, which can be easily implemented in real-time in order to cope with unknown source characteristics, channel dynamics and resource constraints. Our numerical results demonstrate the efficiency of the proposed online algorithm.

On-Line Learning and Optimization for Wireless Video Transmission

In this paper, we address the problem of how to optimize the cross-layer transmission policy for delay-sensitive video streaming over slow-varying flat-fading wireless channels on-line, at transmission time, when the environment dynamics are unknown. We first formulate the cross-layer optimization using a systematic layered Markov decision process (MDP) framework, which complies with the layered architecture of the OSI stack. Subsequently, considering the unknown dynamics of the video sources and underlying wireless channels, we propose a layered real-time dynamic programming (LRTDP) algorithm, which requires no a priori knowledge about the source and network dynamics. LRTDP allows each layer to learn the dynamics on-the-fly, and adjusts its policy autonomously, based on their experienced dynamics as well as limited message exchanges with other layers. Unlike existing cross-layer methods, LRTDP optimizes the cross-layer policy in a layered and on-line fashion, exhibits a low computational complexity, requires limited message exchanges among layers, and is capable to adapt on-the-fly to the experienced environment dynamics. Finally, we prove that LRTDP converges to the optimal cross-layer policy asymptotically. Our numerical experiments show that LRTDP provides comparable performance to the idealized optimal cross-layer solutions based on complete knowledge.

Game-Theoretic Paradigm for Resource Management in Spectrum Agile Wireless Networks

We propose a new way of architecting the wireless multimedia communications systems by jointly optimizing the protocol stack at each station and the resource exchanges among stations. We model wireless stations as rational players competing for available wireless resources in a dynamic repeated game. We investigate and quantify the system performance and the impact of different cross-layer strategies deployed by wireless stations onto their own performance as well as the competing station performance. We show through simulations that the proposed game-theoretic resource management outperforms alternative techniques such as air-fair time and equal time resource allocation in terms of the total system utility