Kao-Shing Hwang

Cooperative multiagent congestion control for high-speed networks

An adaptive multiagent reinforcement learning method for solving congestion control problems on dynamic high-speed networks is presented. Traditional reactive congestion control selects a source rate in terms of the queue length restricted to a predefined threshold. However, the determination of congestion threshold and sending rate is difficult and inaccurate due to the propagation delay and the dynamic nature of the networks. A simple and robust cooperative multiagent congestion controller (CMCC), which consists of two subsystems: a long-term policy evaluator, expectation-return predictor and a short-term rate selector composed of action-value evaluator and stochastic action selector elements has been proposed to solve the problem. After receiving cooperative reinforcement signals generated by a cooperative fuzzy reward evaluator using game theory, CMCC takes the best action to regulate source flow with the features of high throughput and low packet loss rate. By means of learning procedures, CMCC can learn to take correct actions adaptively under time-varying environments. Simulation results showed that the proposed approach can promote the system utilization and decrease packet losses simultaneously.

A REINFORCEMENT LEARNING APPROACH TO CONGESTION CONTROL OF HIGH-SPEED MULTIMEDIA NETWORKS

ABSTRACT A reinforcement learning scheme on congestion control in a high-speed network is presented. Traditional methods for congestion control always monitor the queue length, on which the source rate depends. However, the determination of the congested threshold and sending rate is difficult to couple with each other in these methods. We proposed a simple and robust reinforcement learning congestion controller (RLCC) to solve the problem. The scheme consists of two subsystems: the expectation-return predictor is a long-term policy evaluator and the other is a short-term rate selector, which is composed of action-value evaluator and stochastic action selector elements. RLCC receives reinforcement signals generated by an immediate reward evaluator and takes the best action to control source flow in consideration of high throughput and low cell loss rate. Through on-line learning processes, RLCC can adaptively take more and more correct actions under time-varying environments. Simulation results have shown that the proposed approach can increase system utilization and decrease packet losses simultaneously in comparison with the popular best-effort scheme.

Q-Learning with FCMAC in multi-agent cooperation

In general, Q-learning needs well-defined quantized state spaces and action spaces to obtain an optimal policy for accomplishing a given task. This makes it difficult to be applied to real robot tasks because of poor performance of learned behavior due to the failure of quantization of continuous state and action spaces. In this paper, we proposed a fuzzy-based CMAC method to calculate the contribution of each neighboring state to generate a continuous action value in order to make motion smooth and effective. A momentum term to speed up training has been designed and implemented in a multi-agent system for real robot applications.

Improving the Luminous Efficiency of Red Organic Light-Emitting Diodes Using a Carrier Balance Layer

This work proposes a carrier balance layer (CBL) to yield red organic light-emitting diodes (OLEDs) with high luminous efficiency. In the structure of OLEDs, the CBL (l,3,5-tris(N-phenylbenzimidazol-2,yl)benzene, TPBi) is inserted into the emitting layer, 4-(dicyanomethylene)-2-t-butyl-6(1,1,7,7-tetra-methy-1julolidyl-9-enyl)-4H-pyran (DCJTB)-doped tris(8-hydroxyquinolino)-aluminum (Alq 3 ). Experimental results demonstrated that the luminous efficiency of the OLED with a CBL was 4.63 cd/A 30% higher than that of the typical OLED without a CBL. Inserting CBL into an emitting layer is suggested to rearrange and enlarge the recombination zone. The improvement in efficiency is attributed to the enlargement of the recombination zone in which more excitons were generated and recombined. Sufficient electron-hole recombination improved luminous efficiency.

Self Organizing Decision Tree Based on Reinforcement Learning and its Application on State Space Partition

Most of tree induction algorithms are typically based on a top-down greedy strategy that sometimes makes local optimal decision at each node. Meanwhile, this strategy may induce a larger tree than needed such that requires more redundant computation. To tackle the greedy problem, a reinforcement learning method is applied to grow the decision tree. The splitting criterion is based on long-term evaluations of payoff instead of immediate evaluations. In this work, a tree induction problem is regarded as a reinforcement learning problem and solved by the technique in that problem domain. The proposed method consists of two cycles: split estimation and tree growing. In split estimation cycle, an inducer estimates long-term evaluations of splits at visited nodes. In the second cycle, the inducer grows the tree by the learned long-term evaluations. A comparison with CART on several datasets is reported. The proposed method is then applied to tree-based reinforcement learning. The state spare partition in a critic actor model, adaptive heuristic critic (AHC), is replaced by a regression tree, which is constructed by the proposed method. The experimental results are also demonstrated to show the feasibility and high performance of the proposed system.

Reinforcement learning congestion controller for multimedia surveillance system

The use of reinforcement learning scheme for congestion control in factory surveillance network is presented in this paper. Traditional methods perform congestion control by means of monitoring the queue length. When the queue length is greater than a predefined threshold, the source rate is decreased at a fixed rate. However, the determination of the congested threshold and sending rate is difficult for these methods. We adopted a simple reinforcement learning method, called Adaptive Heuristic Critic (AHC), to solve the problem. The AHC controller maintains an expectation of reward and takes the best policy to control source flow. By way of learning and then taking right actions, simulation results have shown that the approach can promote the system utilization and decrease packet loss.

Cooperation between multiple agents based on partially sharing policy

In human society, learning is essential to intelligent behavior. However, people do not need to learn everything from scratch by their own discovery. Instead, they exchange information and knowledge with one another and learn from their peers and teachers. When a task is too complex for an individual to handle, one may cooperate with its partners in order to accomplish it. Like human society, cooperation exists in the other species, such as ants that are known to communicate about the locations of food and move it cooperatively. Using the experience and knowledge of other agents, a learning agent may learn faster, make fewer mistakes, and create rules for unstructured situations. In the proposed learning algorithm, an agent adapts to comply with its peers by learning carefully when it obtains a positive reinforcement feedback signal, but should learn more aggressively if a negative reward follows the action just taken. These two properties are applied to develop the proposed cooperative learning method conceptually. The algorithm is implemented in some cooperative tasks and demonstrates that agents can learn to accomplish a task together efficiently through a repetitive trials.

A homogeneous agent architecture for robot navigation

A homogeneous agent architecture consists of agents having the same or similar structures in one system. The design introduces the advantage of reducing the development time. In this architecture, the engineer would only design one or few templates for agents, and every agent is constructed by one of these templates. Two categories of agents in the proposed system are the primitive agent and the behavioral agent. The behavioral agents play the role of leaders, so that they can produce output signals through combining the signals of those primitive agents. Hence, we could increase new capability of the whole system flexibly through adding new primitive agents with different capability and adding new behavioral agents to process their opinions. In this paper, we use this homogeneous agent architecture to control a autonomous mobile robot, and it performs well in simulation.

An ARM-Based Q-Learning Algorithm

This article presents an algorithm that combines a FAST-based algorithm (Flexible Adaptable-Size Topology), called ARM, and Q-learning algorithm. The ARM is a self organizing architecture. Dynamically adjusting the size of sensitivity regions of each neuron and adaptively pruning one of the redundant neurons, the ARM can preserve resources (available neurons) to accommodate more categories. The Q-learning is a dynamic programming-based reinforcement learning method, in which the learned action-value function, Q, directly approximates Q*, the optimal action-value function, independent of the policy being followed. In the proposed method, the ARM acts as a cluster to categorize input vectors from the outside world. Clustered results are then sent to the Q-learning architecture in order that it learns to present the best actions to the outside world. The effect of the algorithm is shown through computer simulations of the well-known control of balancing an inverted pendulum on a cart.

Multi-agent congestion control for high-speed networks using reinforcement co-learning

This paper proposes an adaptive reinforcement co-learning method for solving congestion control problems on high-speed networks. Conventional congestion control scheme regulates source rate by monitoring queue length restricted to a predefined threshold. However, the difficulty of obtaining complete statistics on input traffic to a network. As a result, it is not easy to accurately determine the effective thresholds for high-speed networks. We proposed a simple and robust Co-learning Multi-agent Congestion Controller (CMCC), which consists of two subsystems: a long-term policy evaluator and a short-term rate selector incorporated with a co-learning reinforcement signal to solve the problem. The well-trained controllers can adaptively take correct actions to regulate source flow under time-varying environments. Simulation results showed the proposed approach can promote the system utilization and decrease packet losses simultaneously.