Fei Qian

A parallel learning cellular automata for combinatorial optimization problems

Reinforcement learning is a class of learning methodologies in which the controller (or agent) adapts based on external feedback from the random environment. We present a theoretic model of stochastic learning cellular automata (SLCA) as a model of reinforcement learning systems. The SLCA is an extended model of traditional cellular automata, defined as a stochastic cellular automaton with its random environment. There are three rule spaces for the SLCA: parallel, sequential and mixture. We especially study the parallel SLCA with a genetic operator and apply it to the combinatorial optimization problems. The computer simulations of graph partition problems show that the convergence of SLCA is better than the parallel mean field algorithm.

Network load balancing algorithm using ants computing

It is important to reduce the mean transfer time of network, thus the routing algorithm designed to get load balancing is required. In such algorithms, every node must have the following functionalities: getting network traffic information and updating the routing table to reflect traffic information provided by node. In this paper, we describe a routing algorithm for load balancing. Our algorithm deployed ants computing and reinforcement learning. We show that our algorithm achieves load balancing for all nodes of network and works better than other algorithms.

Q-learning automaton

Reinforcement learning is the problem faced by a controller that must learn behavior through trial and error interactions with a dynamic environment. The controllers goal is to maximize reward over time, by producing an effective mapping of states of actions called policy. To construct the model of such systems, we present a generalized learning automaton approach with Q-learning behaviors. Compared to Q-learning, the computational experiments of the pursuit problems show that the proposed reinforcement scheme obtains better results in terms of convergence speed and memory size.

A parallel reinforcement computing model for function optimization problems

Learning Automaton is a learning model with outstanding learning ability, autonomy and guaranteed convergence in the learning process. We propose a parallel computing model with learning automata for function optimization problems and implement it as a sparse distributed parallel computing system. The problems with the traditional reinforcement method using learning automata are: increased difficulty of the adjustment of learning parameters and that of convergence time, with an increased output number. To overcome these problems, we introduce a genetic algorithm (GA) to construct a search space with reduced dimension to search for the optimal output from the entire output space, and provide an efficient way of searching for the smaller-sized search space for the optimal solution. The results of computer simulations verify the usefulness of the proposed method for multivariable function optimization problems.

A Collective Model of Learning Automata with N-Cooperative Random Environments

複数の目標が存在する最適制御においては, 制御システムは, これらの目標の実現を目指す複数の制御者からなる.複数の制約条件のもとで環境の変動が事前に予知できない場合に, 競争的あるいは協力的に共存している各々の制御者が, ある種の集団行動を形成する現象は社会システム, 経済システムなどの中でよく見られる.ゲーム理論の立場から見れば, このようなシステムは協力, 非協力ゲームに帰着される.本研究では, このような複数の目標のもとで動作しているシステムの最適化問題を取り上げ, 複数の静的なランダム環境 (教師) 下で動作している学習オートマトンの集団モデルについて考察し, モデルの定義評価基準などを与え, ランダム環境に適応するための各オートマトンの強化法の構築方法を示す.集団モデルとしては, 種々のタイプが考えられるが, 本文では, とくに, 協力環境下での非協力ゲームのモデルについて考察し, このモデルの静特性および動特性を議論したうえでP-modelとS-modelの二つのタイプの学習オートマトンに対応できる強化法を提案する.また, 強化法の収束性の評明を与え, 簡単なシミュレーション例を用いて, それを検証する