Yi Ouyang

On the Optimality of Myopic Sensing in Multi-State Channels

We consider the channel sensing problem arising in opportunistic scheduling over fading channels, cognitive radio networks, and resource constrained jamming. The same problem arises in many other areas of science and technology as it is an instance of restless bandit problems. The communication system consists of N channels. Each channel is modeled as a multi-state Markov chain. At each time instant a user selects one channel to sense and uses it to transmit information. A reward depending on the state of the selected channel is obtained for each transmission. The objective is to design a channel sensing policy that maximizes the expected total reward collected over a finite or infinite horizon. This problem can be viewed as an instance of restless bandit problems, for which the form of optimal policies is unknown in general. We discover sets of conditions sufficient to guarantee the optimality of a myopic sensing policy; we show that under one particular set of conditions the myopic policy coincides with the Gittins index rule.

Optimal local and remote controllers with unreliable communication

We consider a decentralized optimal control problem for a linear plant controlled by two controllers, a local controller and a remote controller. The local controller directly observes the state of the plant and can inform the remote controller of the plant state through a packet-drop link with acknowledgments. The finite horizon version of this problem was solved in our prior work [1]. The optimal strategies in the finite horizon case were shown to be characterized by coupled Riccati recursions. In this paper, we show that the finite horizon solution is related to the solution of a centralized control problem for an auxiliary Markov jump linear system. By exploiting this connection, we investigate the infinite horizon decentralized control problem. Our main result characterizes a critical threshold Pc for link failure probability above which no decentralized strategy can give a finite cost. When the link failure probability is below this threshold, we provide an explicit characterization of the optimal strategies.

On the optimality of a myopic policy in multi-state channel probing

We consider the channel probing problem arising in opportunistic scheduling over fading channels, cognitive radio networks, and resource constrained jamming. The communication system consists of N channels. Each channel is modeled as a multi-state Markov chain (M.C.). At each time period a user selects one channel to probe and uses it to transmit information. A reward depending on the state of the selected channel is obtained for each transmission. The objective is to design a channel probing policy that maximizes the expected total reward collected over a finite or infinite horizon. This problem can be viewed as an instance of a restless bandit problem, to which the form of optimal policies is unknown in general. We discover conditions sufficient to guarantee the optimality of a myopic probing policy.

On stochastic dynamic games with delayed sharing information structure

We formulate and analyze dynamic games with d-step (d ≥ 1) delayed sharing information structure. The resulting game is a dynamic game of asymmetric information with hidden actions, imperfect observations, and controlled and interdependent system dynamics. We adopt common information based perfect Bayesian equilibrium (CIB-PBE) as the solution concept, and provide a sequential decomposition of the dynamic game. Such a decomposition leads to a backward induction algorithm to compute CIB-PBEs. We discuss the features of our approach to the above class of games and address the existence of CIB-PBEs.

A common information-based multiple access protocol achieving full throughput

We consider a multiple access communication system where multiple users share a common collision channel. In this system, coordination among users is essential to resolve collision issues. However, each system user can only observe its own local traffic and the feedback from the channel. Without a centralized controller, it is challenging to design an efficient coordination protocol. We present a decentralized common information-based multiple access protocol (CIMA). We show that under CIMA collision is totally avoided without channel sensing and the full throughput region of the collision channel is achieved. In addition, simulation results indicate that the CIMA protocol incurs low transmission delay.

Balancing through signaling in decentralized routing

A discrete-time decentralized routing problem in a service system consisting of two service stations and two controllers is investigated. Each controller is affiliated with one station. Each station has an infinite size buffer. Exogenous customer arrivals at each station occur with rate λ. Service times at each station have rate μ. At any time, a controller can route one of the customers waiting in its own station to the other station. Each controller knows perfectly the queue length in its own station and observes the exogenous arrivals to its own station as well as the arrivals of customers sent from the other station. At the beginning, each controller has a probability mass function (PMF) on the number of customers in the other station. These PMFs are common knowledge between the two controllers. A decentralized routing policy that minimizes an infinite horizon average cost per unit time is explicitly determined.

On the existence of near-optimal fixed time control of traffic intersection signals

Traffic intersection signal control usually employ fixed-time, or periodic control policies. Yet, their optimality is unknown. Recent work has proposed queue-dependent max-weight scheduling policies that are optimal. But these are not regarded as practical and unlikely to be adopted. This paper considers the question whether there exist optimal fixed-time signal control policies. We consider a fluid model for arrivals and departures. For simplicity, only one queue is actuated at a time, though this can be generalized. Furthermore, switching the actuation from one queue to the next incurs an all-red Δ delay. We show that there exists a fixed-time signal control policy that is near-optimal. The proof is non-constructive. Nevertheless, it answers an important question and suggests the existence of approximate optimal fixed-time policies that would be practical as well.