Lutao Wang

Rule accumulation method based on Credit Genetic Network Programming

As a new promising evolutionary computation method, Genetic Network Programming (GNP) is good at generating action rules for multi-agent control in dynamic environments. However, some unimportant nodes exist in the program of GNP. These nodes serve as some redundant information which decreases the performance of GNP and the quality of the generated rules. In order to prune these nodes, this paper proposes a novel method named Credit GNP, where a credit branch is added to each node. When the credit branch is visited, the node is neglected and its function is not executed, so that the unimportant nodes could be jumped. The probability of visiting this credit branch and to which node it is jumped is determined by both evolution and Sarsa-learning, therefore, the unimportant nodes could be pruned automatically. Simulation results on the Tile-world problem show that the proposed method could get better programs and generate better and more general rules.

Genetic Network Programming with generalized rule accumulation

Genetic Network Programming(GNP) is a newly developed evolutionary computation method using a directed graph as its gene structure, which is its unique feature. It is competent for dealing with complex problems in dynamic environments and is now being well studied and applied to many real-world problems such as: elevator supervisory control, stock price prediction, traffic volume forecast and data mining, etc. This paper proposes a new method to accumulate evolutionary experiences and guide agents actions by extracting and using generalized rules. Each generalized rule is a state-action chain which contains the past information and the current information. These generalized rules are accumulated and updated in the evolutionary period and stored in the rule pool which serves as an experience set for guiding new agents actions. We designed a two-stage architecture for the proposed method and applied it to the Tile-world problem, which is an excellent benchmark for multi-agent systems. The simulation results demonstrated the efficiency and effectiveness of the proposed method in terms of both generalization ability and average fitness values and showed that the generalized rule accumulation method is especially remarkable when dealing with non-markov problems.

Rule pool updating through Sarsa‐learning to improve adaptability in changing environments

Genetic network programming (GNP) is a new evolutionary algorithm using the directed graph as its chromosome. A GNP-based rule accumulation (GNP-RA) method was proposed previously for multiagent control. However, in changing environments where new situations appear frequently, the old rules in the rule pool become incompetent for guiding the agents actions, and therefore updating them becomes necessary. This paper proposes a more robust rule-based model which can adapt to the environment changes. In order to realize this, Sarsa-learning is used as a tool to update the rules to cope with the unexperienced situations in new environments. Furthermore, Sarsa-learning helps to generate better rules by selecting really important judgments and actions during training. In addition, the e-greedy policy of Sarsa enables GNP-RA to explore the solutions space sufficiently, generating more rules. Simulations on the tile world problem show that the proposed method outperforms the previous ones, namely GP and reinforcement learning.

Genetic Network Programming with updating rule accumulation

Conventional evolutionary computation methods aim to find elite individuals as the optimal solutions. The rule accumulation method tries to find good experiences from individuals throughout the generations and store them as decision rules, which is regarded as solutions. Genetic Network Programming (GNP) is competent for dynamic environments because of its directed graph structure, reusability of nodes and partially observable processes. A GNP based rule accumulation method has been studied and applied to the stock trading problem. However, with the changing of dynamic environments, the old rules in the rule pool are incompetent for guiding agents actions, thus updating these rules becomes necessary. This paper proposes a new method to update the accumulated rules in accordance with the environment changes. Sarsa-learning which is a good on-line learning policy is combined with off-line evolution to generate better individuals and update the rules in the rule pool. Tile-world problem which is an excellent benchmark for multi-agent systems is used as the simulation environment. Simulation results demonstrate the efficiency and effectiveness of the proposed method in dealing with the changing environments.

Genetic Network Programming with new genetic operators

Recently, a new approach named Genetic Network Programming (GNP) has been proposed. GNP can evolve itself and find the optimal solutions. It is based on the ideas of classical evolutionary computation methods such as Genetic Algorithm (GA) and Genetic Programming (GP) and uses the data structure of directed graphs which is the unique feature of GNP. Many studies have demonstrated that GNP can well solve the complex problems in the dynamic environments very efficiently and effectively. As a result, recently, GNP is getting more and more attentions and is being used in many different areas such as data mining, extracting trading rules of stock markets, elevator supervised control systems, etc. Focusing on GNPs distinguished expression ability of the graph structure, this paper proposes an enhanced architecture for the standard GNP in order to improve the performance GNP by using the exploited information extensively during the evolution process of GNP. In the enhanced architecture, we proposed the new genetic operator named Individual Reconstruction which reconstructs and enhances the worst individuals by using the elite information and the crossover and mutation operators of GNP are also modified. In this paper, the proposed architecture has been applied to the tile-world which is an excellent bench mark for evaluating the evolutionary computation architecture. The performance of the new GNP is compared with the conventional GNP. The simulation results show some advantages of the proposed method over the conventional GNPs demonstrating its superiority.

Genetic network programming with route nodes

Many classical methods such as Genetic Algorithm (GA), Genetic Programming (GP), Evolutionary Strategies (ES), etc. have accomplished significant contribution to the study of evolutionary computation. And in the past decade, a new approach named Genetic Network Programming (GNP) has been proposed. It is designed for especially solving complex problems in dynamic environments. Generally speaking, GNP is based on the algorithms of existed classical evolutionary computation techniques and uses the data structure of directed graphs which becomes the unique feature of GNP. So far, many studies have indicated that GNP can solve the complex problems in the dynamic environments very efficiently and effectively. Focusing on GNPs distinguished expression ability of the graph structure, this paper proposes an enhanced architecture for the standard GNP in order to improve the performance of GNP by using the exploited information extensively during the evolution process of GNP. In the enhanced architecture, the important gene information of the elite individuals is extracted and accumulated during evolution. And among the accumulated information, some of them are selected and encapsulated in the Route Nodes which are used to guide the evolution process. In this paper, the proposed architecture has been applied to the tile-world which is an excellent bench mark for evaluating the evolutionary computation architecture. The performance of the GNP with Route Nodes (GNP-RN) is compared with the conventional GNP. The simulation results show some merits of the proposed method over the conventional GNPs demonstrating its superiority.