Yusun Chang

Reinforcement Learning for Repeated Power Control Game in Cognitive Radio Networks

Cognitive radio (CR) users are expected to be uncoordinated users that opportunistically seek the spectrum resource from primary users (PUs) in a competitive way. In most existing works, however, CR users are required to share the interference channel information and power strategies to conduct the game with pricing mechanisms that incur the frequent exchange of information. The requirement of significant communication overheads among CR users impedes fully distributed solutions for the deployment of CR networks, which is a challenging problem in the research communities. In this paper, a robust distributed power control algorithm is designed with low implementation complexity for CR networks through reinforcement learning, which does not require the interference channel and power strategy information among CR users (and from CR users to PUs). To the best of our knowledge, this research provides the solution for the first time for the incomplete-information power control game in CR networks. During the repeated game, CR users can control their power strategies by observing the interference from the feedback signals of PUs and transmission rates obtained in the previous step. This procedure allows achieving high spectrum efficiency while conforming to the interference constraint of PUs. This constrained repeated stochastic game with learning automaton is proved to be asymptotically equivalence to the traditional game with complete information. The properties of existence, diagonal concavity and uniqueness for the game are studied. A Bush-Mosteller reinforcement learning procedure is designed for the power control algorithm, and the properties of convergence and learning rate of the algorithm are analyzed. The performance of the learning-based power control algorithm is thoroughly investigated with simulation results, which demonstrates the effectiveness of the proposed algorithm in solving variety of practical CR network problems for real-world applications.

Goodput Enhancement of VANETs in Noisy CSMA/CA Channels

The growing interest in vehicular ad hoc networks (VANETs) enables decentralized traveler information systems to become more feasible and effective in Intelligent Transportation Systems (ITS). Major challenges in such network environments include varying path characteristics and vulnerable channel quality resulting from dynamic traffic conditions and the design of the road. This paper demonstrates a feasible methodology that can enhance inter-vehicle information dissemination using dynamic optimal fragmentation with rate adaptation algorithm (DORA). DORA achieves maximum goodput in wireless mobile networks by computing a fragmentation threshold and transmitting optimal sized packets with maximum transfer rates. To estimate the SNR in the model, an adaptive on-demand UDP estimator is designed to reduce estimation overhead. Several test-beds were developed to evaluate the performance of DORA in channel estimation accuracy, ad hoc network goodput, and vehicle-to-vehicle network goodput along I-85 in Atlanta, Georgia. The proposed algorithm is an energy-efficient, generic CSMA/CA MAC protocol for wireless mobile computing applications, and enhances system goodput in ad hoc networks and vehicle-to-vehicle networks without modification of the base protocols.

Dynamic Optimal Fragmentation for Goodput Enhancement in WLANs

To meet the demand for broadband wireless communication, wireless systems should work well in typical wireless environments, characterized by the path loss of the signals, multipath fading, interference to adjacent channels, and random errors. IEEE 802.11 VVLANs use the unlicensed 2.4 GHz industrial, scientific and medical (ISM) band, which is vulnerable to noise generated by TVs, microwaves, and cordless phones. This paper proposes an algorithm to enhance system goodput through the dynamic optimal fragmentation. The number of contending stations, packet collisions, packet error probabilities, and fragmentation overheads are modeled in the analysis. Using an adaptive SNR estimator, the sender estimates the SNR of the receiver, and chooses a fragmentation threshold to shape arbitrary sized packets into optimal length packets. Through the rigorous analysis and extensive experiments with implemented test-bed, we show that the dynamic optimal fragmentation enhances the goodput approximately 18% in a typical WLAN environment. The experiment results reinforce that the algorithm is a comprehensive analytical model applicable to any CSMA/CA based MAC protocol for next generation wireless networks, and a realistic approach that can be deployed without changing the IEEE802.11 MAC protocol.

Handshaking vs. instant broadcast in VANET safety message routing

Vehicular ad hoc networks (VANETs) rely on multi-hop communication in order to disseminate information to locations beyond the transmission range. In this paper, the use of handshaking mechanism is evaluated against instant transmission for safety message routing in VANETs for the first time. It is argued that due to the small size of VANET safety message (mean payload size of 100 bytes), handshaking may not be advantageous to avoid hidden node problem and may incur overhead delay. The simplified instant broadcast routing without the use of handshake mechanism is recommended. Theoretical analysis of message propagation delay using instant broadcast routing is shown which can be easily extended for any broadcast method. Two prominent handshake based broadcast routing schemes are implemented in ns-3 along with instant broadcast scheme. The simulation results reveal that instant broadcast effectively lowers the message propagation delay by 100% each hop and guarantees reliable dissemination of safety messages to all endangered vehicles on the road.

Dynamic Service-Channels Allocation (DSCA) in vehicular ad-hoc networks

Providing tools to achieve a high level of safety transportation is the important objective in VANETs research. Hence, most of the works are devoted to developing safety-message dissemination algorithms. However, non-safety applications can also contribute to the network efficiency by exchanging traffic information. To support these applications, VANETs adopt the wireless access in the vehicular environment (WAVE) to guarantee the quality of service (QoS) using four different access categories. However, the WAVE could not provide the QoS to the users due to the collisions caused by the small contention-window size for the top priority traffic when top-priority traffic is dominant. Therefore, this small contention-window size setting for the prioritization induces severe performance degradation. In this paper, the Dynamic Service-Channels Allocation (DSCA) method is proposed to maximize throughput by dynamically assigning different service channels to the users. Theoretical analysis and extensive simulation works suggest that the DSCA improves the average throughput by 22% over the uniform channel allocation method. Moreover, the DSCA can be easily implemented in real devices without any protocol modifications.

Performance Optimization of a Contention Based Broadcasting Algorithm in VANETs

Sharing emergency messages amongst vehicles on the road can greatly reduce traffic accidents. In Vehicular Ad Hoc Networks (VANETs) to share these emergency messages, fast and reliable message dissemination is the key objective in a highly dynamic VANET environment. This research proposes a novel broadcasting protocol to achieve these goals by introducing handshake-less broadcasting, ACK Decoupling and efficient collision resolution mechanism. A key contribution of this paper is to analyze and optimize the broadcasting algorithm using thorough mathematical modeling with extensive simulations. With the suggested optimization process, the message propagation delay can be significantly reduced.

Learning through Reinforcement for Repeated Power Control Game in Cognitive Radio Networks

This paper studies the repeated power control game in cognitive radio (CR) networks through reinforcement learning without channel and power strategy information exchange among CR users. Unlike traditional game-theoretical approaches on CR power control, this research solves the incomplete information power control problems for selfish and autonomous CR users for the first time. Each CR user in the problem only knows its own channel and power strategy while the information of primary users (PUs) and other different types of CR users are unknown. The formulated power control problem is a constrained repeated stochastic game with learning automaton. The objective of this repeated game is to maximize the average utility of each CR user under the interference power constraints of PUs. At each time step, the CR user only knows its own utility and the interference functions after the play but no further information. This power control game is proved to be asymptotically equivalent to the traditional game theory approaches. The properties of existence, diagonal concavity and uniqueness for this game are illustrated in detail. A Bush-Mosteller reinforcement learning procedure is designed for the power control algorithm. Finally, the learning based power control algorithm is implemented, and the simulation results with detailed analysis are shown to enforce the effectiveness of the proposed algorithms.

An Adaptive On-Demand Channel Estimation for Vehicular Ad Hoc Networks

Various vehicular communication technologies have been proposed to provide reliable and seamless wireless communications. Short-range vehicular communications can be established using IEEE 802.11p, and the range of these protocols can be extended using mobile multi-hop ad hoc networks. To provide consistent Quality of Service (QoS) with maximum achievable transmission rate, many algorithms assume that the Signal to Noise Ratio (SNR) of the receiver is known, or can be obtained by modifying protocols. In this paper, we present an efficient on-demand adaptive channel estimation technique. Using simple UDP messages without changing protocols, the estimator predicts the SNR with minimum signaling overheads. Extensive experiments in vehicular environments along the I-85 highway and residential areas in Atlanta, Georgia reinforce that the proposed algorithm is efficient and reliable channel estimation technique for next generation wireless mobile networks.

A Security Scheme for Centralized Scheduling in IEEE 802.16 Mesh Networks

Worldwide interoperability for microwave access (WiMAX), designed by the IEEE 802.16 standards group, can deliver high-speed connectivity for both fixed and vehicular speed subscribers. Mesh networking support was added for improved coverage and throughput. In order to operate efficiently, communication must be scheduled either by a distributed, centralized, or hybrid algorithm. However, in the current standard, there are problems that need addressing: malicious sponsor nodes, privacy, and link-by-link encryption load. We propose a new security scheme for centralized scheduling, which not only provides a simple way for mutual authentication between a joining node and the sponsor node, but also supports better privacy and reduces the encryption load.

Goodput optimization in CSMA/CA wireless networks

Packet based wireless networks have provided high data rates, and the desire for ubiquitous wireless computing is growing rapidly. To accommodate these demands, wireless systems should work well in typical wireless environments. The most widely deployed WLAN technology, IEEE 802.11, uses the unlicensed ISM band, which is vulnerable to noise generated by electronic devices. This paper proposes a way to enhance system goodput in CSMA/CA MAC protocols using fragmentation. Our analytical model considers the number of contending stations, packet collisions, packet error probabilities, and fragmentation overheads. With extensive experiments in WLAN and ad hoc networks, we show the optimization improves the goodput approximately 18% in a typical WLAN and 48% in ad hoc networks. The proposed approach can be implemented to any CSMA/CA MAC protocol without modifying the standard.