Qing Zhao

Sensor networks with mobile agents

Architecture for large scale low power sensor network is proposed. Referred to as sensor networks with mobile agents (SENMA), SENMA exploit node redundancies by introducing mobile agents that communicate opportunistically with a large field of sensors. The addition of mobile agents shifts computationally intensive tasks away from primitive sensors to more powerful mobile agents, which enables energy efficient operations under severely limited power constraints. An opportunistic ALOHA random access coupled with a direct sequence spread spectrum physical layer is proposed. A comparison of SENMA with a flat ad hoc sensor network shows a substantial gain in energy efficiency.

Indexability of Restless Bandit Problems and Optimality of Whittle Index for Dynamic Multichannel Access

In this paper, we consider a class of restless multiarmed bandit processes (RMABs) that arises in dynamic multichannel access, user/server scheduling, and optimal activation in multiagent systems. For this class of RMABs, we establish the indexability and obtain Whittle index in closed form for both discounted and average reward criteria. These results lead to a direct implementation of Whittle index policy with remarkably low complexity. When arms are stochastically identical, we show that Whittle index policy is optimal under certain conditions. Furthermore, it has a semiuniversal structure that obviates the need to know the Markov transition probabilities. The optimality and the semiuniversal structure result from the equivalence between Whittle index policy and the myopic policy established in this work. For nonidentical arms, we develop efficient algorithms for computing a performance upper bound given by Lagrangian relaxation. The tightness of the upper bound and the near-optimal performance of Whittle index policy are illustrated with simulation examples.

A multiqueue service room MAC protocol for wireless networks with multipacket reception

An adaptive medium-access control (MAC) protocol for heterogeneous networks with finite population is proposed. Referred to as the multiqueue service room (MQSR) protocol, this scheme is capable of handling users with different quality-of-service (QoS) constraints. By exploiting the multipacket reception (MPR) capability, the MQSR protocol adaptively grants access to the MPR channel to a number of users such that the expected number of successfully received packets is maximized in each slot. The optimal access protocol avoids unnecessary empty slots for light traffic and excessive collisions for heavy traffic. It has superior throughput and delay performance as compared to, for example, the slotted ALOHA with the optimal retransmission probability. This protocol can be applied to random-access networks with multimedia traffic.

Sensor placement for maximizing lifetime per unit cost in wireless sensor networks

Lifetime per unit cost, defined as the network lifetime divided by the number of sensors deployed in the network, can be used to measure the utilization efficiency of sensors in a wireless sensor network (WSN). Analyzing the lifetime per unit cost of a linear WSN, we find that deploying either an extremely large or an extremely small number of sensors is inefficient in terms of lifetime per unit cost. We thus seek answers to the following questions: how many sensors should be deployed and how to deploy them to maximize the lifetime per unit cost. Numerical and simulation results are provided to study the optimal sensor placement and the optimal number of deployed sensors

Learning in a Changing World: Restless Multiarmed Bandit With Unknown Dynamics

We consider the restless multiarmed bandit problem with unknown dynamics in which a player chooses one out of N arms to play at each time. The reward state of each arm transits according to an unknown Markovian rule when it is played and evolves according to an arbitrary unknown random process when it is passive. The performance of an arm selection policy is measured by regret, defined as the reward loss with respect to the case where the player knows which arm is the most rewarding and always plays the best arm. We construct a policy with an interleaving exploration and exploitation epoch structure that achieves a regret with logarithmic order. We further extend the problem to a decentralized setting where multiple distributed players share the arms without information exchange. Under both an exogenous restless model and an endogenous restless model, we show that a decentralized extension of the proposed policy preserves the logarithmic regret order as in the centralized setting. The results apply to adaptive learning in various dynamic systems and communication networks, as well as financial investment.

Distributed opportunistic transmission for wireless sensor networks

We consider protocol design for extracting information from sensors by a mobile access point. Energy efficiency, defined as the expected number of bits reliably received for each unit of energy consumed, is used as the performance measure. A distributed opportunistic information retrieval protocol which exploits channel state information (CSI) is proposed. Referred to as the CSI-based carrier sensing, this protocol encodes the channel state into the backoff strategy of carrier sensing. When the propagation delay is negligible, CSI-based carrier sensing achieves the highest energy efficiency of the opportunistic strategy. For significant propagation, we construct the backoff function which maps the channel state to backoff time to minimize the performance loss. The CSI-based carrier sensing with the constructed backoff strategy is shown to be robust to propagation delay.

Quality-of-service specific information retrieval for densely deployed sensor networks

A new MAC protocol is proposed for the reachback operation in large scale, densely deployed sensor networks. Referred to as quality-of-service specific information retrieval (QUIRE), the proposed protocol aims to assure QoS requirement with a minimum amount of transmissions from sensors. By enabling only one sensor in a neighborhood to transmit, QUIRE ensures that the data access point receives no redundant information for reconstructing the sensed field within a given maximum distortion (QoS). It jointly minimizes system latency (the amount of time spent for data collection) and total energy consumption (the total number of transmissions from sensors).

Network configuration for optimal utilization efficiency of wireless sensor networks

This paper addresses the problem of configuring wireless sensor networks (WSNs). Specifically, we seek answers to the following questions: how many sensors should be deployed, what is the optimal sensor placement, and which transmission structure should be employed. The design objective is utilization efficiency defined as network lifetime per unit deployment cost. We propose an optimal approach and an approximation approach with reduced complexity to network configuration. Numerical and simulation results demonstrate the near optimal performance of the approximation approach. We also study the impact of sensing range, channel path loss exponent, sensing power consumption, and event arrival rate on the optimal network configuration.

Retail pricing for stochastic demand with unknown parameters: An online machine learning approach

The problem of dynamically pricing of electricity by a retailer for customers in a demand response program is considered. It is assumed that the retailer obtains electricity in a two-settlement wholesale market consisting of a day ahead market and a real-time market. Under a day ahead dynamic pricing mechanism, the retailer aims to learn the aggregated demand function of its customers while maximizing its retail profit. A piecewise linear stochastic approximation algorithm is proposed. It is shown that the accumulative regret of the proposed algorithm grows with the learning horizon T at the order of O(log T). It is also shown that the achieved growth rate cannot be reduced by any piecewise linear policy.

Active Hypothesis Testing for Anomaly Detection

The problem of detecting a single anomalous process among a finite number M of processes is considered. At each time, a subset of the processes can be observed, and the observations from each chosen process follow two different distributions, depending on whether the process is normal or abnormal. The objective is a sequential search strategy that minimizes the expected detection time subject to an error probability constraint. This problem can be considered as a special case of active hypothesis testing first considered by Chernoff where a randomized strategy, referred to as the Chernoff test, was proposed and shown to be asymptotically (as the error probability approaches zero) optimal. For the special case considered in this paper, we show that a simple deterministic test achieves asymptotic optimality and offers better performance in the finite regime. We further extend the problem to the case where multiple anomalous processes are present. In particular, we examine the case where only an upper bound on the number of anomalous processes is known.