Thai Duong

Location Assisted Coding (LAC): Embracing Interference in Free Space Optical Communications

As the number of wireless devices grows, the increasing demand for the shared radio frequency (RF) spectrum becomes a critical problem. Unlike wired communications in which, theoretically, more fibers can be used to accommodate the increasing bandwidth demand, wireless spectrum cannot be arbitrarily increased due to the fundamental limitations imposed by the physical laws. On the other hand, recent advances in free space optical (FSO) technologies promise a complementary approach to increase wireless capacity. However, high-speed FSO technologies are currently confined to short distance transmissions, resulting in limited mobility. In this paper, we briefly describe WiFO, a hybrid WiFi-FSO network for Gbps wireless local area network (WLAN) femtocells that can provide up to one Gbps per user while maintaining seamless mobility. While typical RF femtocells are non-overlapped to minimize inter-cell interference, there are advantages of using overlapped femtocells to increase mobility and throughput when the number of users is small. That said, the primary contribution of this paper will be a novel location assisted coding (LAC) technique used in the WiFO network that aims to increase throughput and reduce interference for multiple users in a dense array of femtocells. Both theoretical analysis and numerical experiments show orders of magnitude increase in throughput using LAC over basic codes.

Virtual Machine Placement via Q-Learning with Function Approximation

While existing virtual machine technologies provide easy-to-use platforms for distributed computing applications, many are far from efficient and not designed to accommodate diverse objectives, which dramatically penalizes their performance. These shortcomings arise from 1) not having a formal optimization framework that readily leads to algorithmic solutions for diverse objectives; 2) not incorporating the knowledge of the underlying network topologies and the communication/interaction patterns among the virtual machines/services, and 3) not considering the time-varying aspects of real-world environments. This paper formalizes an optimization framework and develops corresponding algorithmic solutions using Markov Decision Process and Q-Learning for virtual machine/service placement and migration for distributed computing in time-varying environments. Importantly, the knowledge of the underlying topologies of the computing infrastructure, the interaction patterns between the virtual machines, and the dynamics of the supported applications will be formally characterized and incorporated into the proposed algorithms in order to improve performance. Simulation results for small-scale and large-scale networks are provided to verify our solution approach.

Network Protocol Designs: Fast Queuing Policies via Convex Relaxation

With the recent rise of mobile and multimedia applications, other considerations such as power consumption and/or Quality of Service (QoS) are becoming increasingly important factors in designing network protocols. As such, we present a new framework for designing robust network protocols under varying network conditions that attempts to integrate various given objectives while satisfying some pre-specified levels of Quality of Service. The proposed framework abstracts a network protocol as a queuing policy, and relies on convex relaxation methods and the theory of mixing time for finding the fast queuing policies that drive the distribution of packets in a queue to a given target stationary distribution. In addition, we show how to augment the basic proposed framework to obtain a queuing policy that produces ε-approximation to the target distribution with faster convergence time which is useful in fast-changing network conditions. Both theoretical and simulation results are presented to verify the effectiveness of the proposed framework.

Data Collection in Sensor Networks via the Novel Fast Markov Decision Process Framework

We investigate the data collection problem in sensor networks. The network consists of a number of stationary sensors deployed at different sites for sensing and storing data locally. A mobile element moves from site to site to collect data from the sensors periodically. There are different costs associated with the mobile element moving from one site to another, and different rewards for obtaining data at different sensors. Furthermore, the costs and the rewards are assumed to change abruptly. The goal is to find a “fast” optimal movement pattern/policy of the mobile element that optimizes for the costs and rewards in non-stationary environments. We formulate and solve this problem using a novel optimization framework called fast Markov decision process (FMDP). The proposed FMDP framework extends the classical Markov decision process theory by incorporating the notion of mixing time that allows for the trade-off between the optimality and the convergence rate to the optimality of a policy. Theoretical and simulation results are provided to verify the proposed approach.

Adiabatic Markov Decision Process with application to queuing systems

Markov Decision Process (MDP) is a well-known framework for devising the optimal decision making strategies under uncertainty. Typically, the decision maker assumes a stationary environment which is characterized by a time-invariant transition probability matrix. However, in many realworld scenarios, this assumption is not justified, thus the optimal strategy might not provide the expected performance. In this paper, we study the performance of the classic Value Iteration (VI) algorithm for solving an MDP problem under non-stationary environments. Specifically, the non-stationary environment is modeled as a sequence of time-variant transition probability matrices governed by an adiabatic evolution inspired from quantum mechanics. We characterize the performance of the VI algorithm subject to the rate of change of the underlying environment. The performance is measured in terms of the convergence rate to the optimal average reward. We show two examples of queuing systems that make use of our analysis framework.

Location-Assisted Coding for FSO Communication

Recent years have witnessed an explosive growth in the number of wireless devices. This development has fueled much research in wireless access technologies to efficiently use radio frequency spectrum. On the other hand, recent advances in free space optical (FSO) technologies promise a complementary approach to increase wireless capacity. In this paper, we describe WiFO, a hybrid WiFi and FSO high-speed wireless local area network of femtocells that can provide high bit rates while maintaining seamless mobility. Importantly, we introduce a novel location-assisted coding (LAC) technique, based on which, the number of novel rate allocation algorithms is proposed to increase throughput and reduce interference for multiple users in a dense array of overlapped femtocells. Both theoretical analysis and numerical results show orders of magnitude increase in throughput using LAC over existing schemes for various random topologies.

Fast Markov Decision Process for data collection in sensor networks.

We investigate the data collection problem in sensor networks. The network consists of a number of stationary sensors deployed at different sites for sensing and storing data locally. A mobile element moves from sites to sites to collect data from the sensors periodically. There are different costs associated with the mobile element moving from one site to another, and different rewards for obtaining data at different sensors. Furthermore, the costs and the rewards are assumed to change abruptly. The goal is to find a “fast” optimal movement pattern/policy of the mobile element that optimizes for the costs and rewards in non-stationary environments. We propose a novel optimization framework called Fast Markov Decision Process (FMDP) to solve this problem. The proposed FMDP framework extends the classical Markov Decision Process theory by incorporating the notion of mixing time that allows for the tradeoff between the optimality and the convergence rate to the optimality of a policy. Theoretical and simulation results are provided to verify the proposed approach.

Achieving Quality of Service via packet distribution shaping

Conventional Quality of Service (QoS) for multimedia networking applications are typically specified by a certain set of requirements on latency, jitter, bandwidth, and packet loss rate. In this paper, we introduce a novel approach to QoS via the notion of distribution shaping in which a pre-specified distribution of packets in a queue is achieved via queuing policies. In a way, the distribution-based QoS is more general since the distribution of packets in the queue captures all the statistical information regarding the throughput, latency, delay jitter, and packet loss rate. We present a convex optimization framework for obtaining the optimal queueing policy that drives any initial distribution of packets in the queue to the desired distribution in the fastest time. We then augment the proposed framework to obtain a queueing policy that produces ∈-approximation to the target distribution with even faster convergence time. The augmented framework is useful in dynamic settings where traffic statistics change frequently, and thus fast adaptation is preferable. Both simulation and theoretical results are provided to verify our approach.

Integrating free-space optical communication links with existing WiFi (WiFO) network

Recently, free-space optical (FSO) systems have generated great interest due to their large bandwidth potential and a line-of-sight physical layer of protection. In this paper, we propose WiFO, a novel hybrid system, FSO downlink and WiFi uplink, which will integrate currently available WiFi infrastructure with inexpensive infrared light emitting diodes. This system takes full advantage of the mobility inherent in WiFi networks while increasing the downlink bandwidth available to each end user. We report the results of our preliminary investigation that show the capabilities of our prototype design in terms of bandwidth, bit error rates, delays and transmission distances.

Adiabatic Markov Decision Process: Convergence of Value Iteration Algorithm

Markov decision process (MDP) is a well-known framework for devising the optimal decision-making strategies under uncertainty. Typically, the decision maker assumes a stationary environment which is characterized by a time-invariant transition probability matrix. However, in many real-world scenarios, this assumption is not justified, thus the optimal strategy might not provide the expected performance. In this paper, we study the performance of the classic value iteration algorithm for solving an MDP problem under nonstationary environments. Specifically, the nonstationary environment is modeled as a sequence of time-variant transition probability matrices governed by an adiabatic evolution inspired from quantum mechanics. We characterize the performance of the value iteration algorithm subject to the rate of change of the underlying environment. The performance is measured in terms of the convergence rate to the optimal average reward. We show two examples of queuing systems that make use of our analysis framework.