Hitoshi Iima

A new encoding scheme for solving job shop problems by genetic algorithm

A new encoding scheme in genetic algorithms is proposed by using an ordering string for the classic job shop scheduling problem. A genetic algorithm is designed for searching the semi-active schedule space, which is encoded by the string space. A new crossover, set-partition crossover is introduced in accompanying the genetic searching. An associated selection strategy and a production structure are properly established for this fashion of encoding. This encoding scheme naturally overcomes the infeasibility problem in genetic iterations. Experiments show that the proposed genetic algorithm is effective, and optimal solutions are attainable in some probability for Fisher and Thompson problems with definite hardness.

Models of hopfield-type clifford neural networks and their energy functions - hyperbolic and dual valued networks -

Recently, models of neural networks in the real domain have been extended into the high dimensional domain such as the complex and quaternion domain, and several high-dimensional models have been proposed. These extensions are generalized by introducing Clifford algebra (geometric algebra). In this paper we extend conventional real-valued models of recurrent neural networks into the domain defined by Clifford algebra and discuss their dynamics. We present models of fully connected recurrent neural networks, which are extensions of the real-valued Hopfield type neural networks to the domain defined by Clifford algebra. We study dynamics of the models from the point view of existence conditions of an energy function. We derive existence conditions of an energy function for two classes of the Hopfield type Clifford neural networks.

Reinforcement Learning through Interaction among Multiple Agents

In ordinary reinforcement learning algorithms, a single agent learns to achieve a goal through many episodes. If a learning problem is complicated, it may take a much computation time to obtain the optimal policy. Meanwhile, for optimization problems, multi-agent search methods such as particle swarm optimization have been recognized that they are able to find rapidly a global optimal solution for multi-modal functions with wide solution space. This paper proposes a reinforcement learning algorithm by using multiple agents. In this algorithm, the multiple agents learn through not only their respective experiences but also interaction among them. For the interaction methods this paper proposes three strategies: the best action-value strategy, the average action-value strategy and the particle swarm strategy

Swarm reinforcement learning method based on ant colony optimization

In ordinary reinforcement learning methods, a single agent learns to achieve a goal through many episodes. Since the agent essentially learns by trial and error, it takes much computation time to acquire an optimal policy especially for complicated learning problems. Meanwhile, for optimization problems, population-based methods such as particle swarm optimization have been recognized that they are able to find rapidly the global optimal solution for multi-modal functions with wide solution space. We recently proposed swarm reinforcement learning methods in which multiple agents are prepared and they learn through not only their respective experiences but also exchanging information among them. In these methods, it is important how to design a method of exchanging the information. In this paper, we propose a swarm reinforcement learning method based on ant colony optimization, which is an optimization method inspired from behavior of real ants using trail pheromones, in order to acquire the optimal policy rapidly even for complicated reinforcement learning problems. In the proposed method, the agents exchange their information through Pheromone-Q values which we define so as to make them play the same role as the trail pheromones. The proposed method is applied to shortest path problems, and its performance is demonstrated through numerical experiments.

Autonomous decentralized scheduling algorithm for a job-shop scheduling problem with complicated constraints

This paper deals with a job-shop scheduling problem with complicated constraints. In the problem precedence relations exist nor only among operations but also among jobs. Furthermore, this production system has several types of single function machines and a type of multifunction machine, and the number of machines available is plural for the respective types. Therefore selection of the machine is necessary for executing each operation. An autonomous decentralized scheduling algorithm is proposed to obtain a suboptimal solution of this problem. The effectiveness of the proposed algorithm is investigated by examining numerical results. Moreover, the algorithm is also applied to a rescheduling problem in the case where a machine breaks down.

Swarm reinforcement learning algorithms based on particle swarm optimization

In ordinary reinforcement learning algorithms, a single agent learns to achieve a goal through many episodes. If a learning problem is complicated, it may take much computation time to acquire the optimal policy. Meanwhile, for optimization problems, population-based methods such as particle swarm optimization have been recognized that they are able to find rapidly the global optimal solution for multi-modal functions with wide solution space. We recently proposed reinforcement learning algorithms in which multiple agents are prepared and they learn through not only their respective experiences but also exchanging information among them. In these algorithms, it is important how to design a method of exchanging the information. This paper proposes some methods of exchanging the information based on the update equations of particle swarm optimization. The proposed algorithms using these methods are applied to a shortest path problem, and their performance is compared through numerical experiments.

Swarm reinforcement learning algorithms based on Sarsa method

We recently proposed swarm reinforcement learning algorithms in which multiple agents are prepared and they all learn concurrently with two learning strategies: individual learning and learning through exchanging information. In the proposed swarm reinforcement learning algorithms, Q-learning method was used for the individual learning. However, there have been proposed several reinforcement learning methods, and it is required to investigate how to apply these methods to swarm reinforcement learning algorithms and evaluate their performance. In this paper, we propose swarm reinforcement learning algorithms based on Sarsa method in order to obtain an optimal policy rapidly for problems with negative large rewards. The proposed algorithm is applied to a shortest path problem, and its performance is examined through numerical experiments.

Proposition of selection operation in a genetic algorithm for a job shop rescheduling problem

This paper deals with a two-objective rescheduling problem in a job shop for alteration of due date. One objective of this problem is to minimize the total tardiness, and the other is to minimize the difference of schedule. A genetic algorithm is proposed, and a new selection operation is particularly introduced to obtain the Pareto optimal solutions in the problem. At every generation in the proposed method, two solutions are picked up as the parents. While one of them is picked up from the population, the other is picked up from the archive solution set. Then, two solutions are selected from these parents and four children generated by means of the crossover and the mutation operation. The candidates selected are not only solutions close to the Pareto-optimal front but also solutions with a smaller value of the total tardiness, because the initial solutions are around the solution in which the total tardiness is zero. For this purpose, the solution space is ranked on the basis of the archive solutions. It is confirmed from the computational result that the proposed method outperforms other methods.

Application of genetic algorithm to a large-scale scheduling problem for a metal mold assembly process

A genetic algorithm is applied to an optimal scheduling problem for a metal mold assembly process. The process is operated basically in a job shop mode with additional constraints, such as the precedence constraints among jobs and the allocation of each operation to different kinds of parallel machines. The objective of the problem is to minimize the sum of the tardiness of each job. Furthermore, the problem is large-scale because of a long scheduling period. In the design of genetic algorithm an individual description and genetic operators are proposed to satisfy the constraints. A method for decomposing the set of jobs is also proposed to cope with the large-scale problem. In order to examine the effectiveness of the algorithm, a simulation is carried out on the basis of large-scale operation data.

Application of genetic algorithm to scheduling problems in manufacturing processes

A genetic algorithm approach to optimal scheduling in manufacturing processes is considered. The scheduling problem is described as a production ordering problem with many constraints. Various methods of the individual description and the decoding rule are presented to improve the feasibility of the solution. A numerical result is shown for a modified flowshop scheduling problem.