Harukazu Igarashi

An estimation of parameters in an energy function used in a simulated annealing method

When a combinatorial optimization problem such as the traveling salesman problem is solved by a simulated annealing method, it is common to use an energy function that consists of two kinds of terms: a cost term which should be minimized and a constraint term which expresses constraints imposed on solutions. The author proposes a method for determining appropriate values of weights of constraint terms in the annealing process. If appropriate values of parameters expressing the weights of the constraint terms are not given, only solutions which do not satisfy all constraints, or high-cost solutions, can be found. A method is presented that leads to appropriate values of these parameters and finds an optimal solution in a systematic manner. The method was applied to 10-city traveling salesman problems, and the experiments showed the effectiveness of the method.<<ETX>>

Learning of soccer player agents using a policy gradient method: Coordination between kicker and receiver during free kicks

The RoboCup Simulation League is recognized as a test bed for research on multi-agent learning. As an example of multi-agent learning in a soccer game, we dealt with a learning problem between a kicker and a receiver when a direct free kick is awarded just outside the opponentpsilas penalty area. In such a situation, to which point should the kicker kick the ball? We propose a function that expresses heuristics to evaluate an advantageous target point for safely sending/receiving a pass and scoring. The heuristics includes an interaction term between a kicker and a receiver to intensify their coordination. To calculate the interaction term, we let kicker/receiver agents have a receiver/kicker action decision model to predict his teammatepsilas action. The evaluation function makes it possible to handle a large space of states consisting of the positions of a kicker, a receiver, and their opponents. The target point of the free kick is selected by the kicker using Boltzmann selection with an evaluation function. Parameters in the function can be learned by a kind of reinforcement learning called the policy gradient method. The point to which a receiver should run to receive the ball is simultaneously learned in the same manner. The effectiveness of our solution was shown by experiments.

Renormalized Field Theory of Random Magnetic Mixtures with Competing Spin Anisotropies. I

Les points fixes des fonctions β et les valeurs propres de la matrice B de la partie I sont calcules dans la partie II. On confirme que les 256 points fixes des melanges du type Ising-XY non symetriques sont instables et que les 128 points fixes des melanges du type Ising-Ising non symetriques deviennent instables. Les points tetracritiques des systemes magnetiques etudies ne sont pas du second ordre

A solution for combinational optimization problems using a two-layer random field model-mean-field approximation

In the solution of the combinational optimization problem such as the traveling salesman problem, the usual approach is to define the energy function, which consists of the term representing the cost to be minimized and the terms representing the constraint for the solution. It is important at this stage to define adequately the weight coefficients for the constraint terms. For this purpose, a solution method based on the two-layer random field model has already been proposed. However, it is desirable from the viewpoint of the processing speed to apply the deterministic annealing to the analog neuron system obtained by the mean-ield approximation, rather than to apply directly the simulated annealing to the binary neuron system. In his case, it is important also to define adequately the weight coefficients in the energy function. This paper considers the already proposed method which automatically adjusts the weight coefficients using the two-layer random field model. An elaboration is presented which applies the method to the search of the optimal solution by the deterministic nnealing. In this study, the connection machine CM-2), which is a SIMD-type parallel computer, is used to handle the relatively large-scale problem composed of 64 cities.

Path-planning and navigation of a mobile robot as discrete optimization problems

There is huge diversity among navigation and path-planning problems in the real world because of the enormous number and great variety of assumptions about the environments, constraints, and tasks imposed on a robot. To deal with this diversity, we propose a new solution to the path-planning and navigation of a mobile robot. In our approach, we formulated the following two problems at each time-step as discrete optimization problems: (1) estimation of a robots location, and (2) action decision. For the first problem, we minimize an objective function that includes a data term, a constraint term, and a prediction term. This approach is an approximation of Markov localization. For the second problem, we define and minimize another objective function that includes a goal term, a smoothness term, and a collision term. Simulation results show the effectiveness of our approach.

Individual Tactical Play and Action Decision Based on a Short-Term Goal - Team Descriptions of Team Miya and Team Niken

In this paper we present descriptions of our two teams that participated in the simulator league of RoboCup 97. One of the teams is characterized by soccer agents that make individual tactical plays without communicating with each other. The other team is characterized by the use of an action-decision algorithm based on a short-term goal and current information. The two teams were among the best of 8 and 16 teams at the competition.

Low temperature thermodynamics of quantum spin chains

Abstract Low temperature thermodynamic quantities of quantum spin chains are obtained by combining the high temperature expansion and an argument based on the spin wave theory. The method is tested in the XY chain and applied to the Heisenberg chain for which low temperature specific heat and antiferromagnetic susceptibility are calculated.

Behavior Learning Based on a Policy Gradient Method: Separation of Environmental Dynamics and State Values in Policies

Policy gradient methods are very useful approaches in reinforcement learning. In our policy gradient approach to behavior learning of agents, we define an agents decision problem at each time step as a problem of minimizing an objective function. In this paper, we give an objective function that consists of two types of parameters representing environmental dynamics and state-value functions. We derive separate learning rules for the two types of parameters so that the two sets of parameters can be learned independently. Separating these two types of parameters will make it possible to reuse state-value functions for agents in other different environmental dynamics, even if the dynamics is stochastic. Our simulation experiments on learning hunter-agent policies in pursuit problems show the effectiveness of our method.

Navigation of a Mobile Robot Formulated in Terms of Discrete Optimization Problems

The problem of navigation of an autonomous mobile robot has a great variety of solutions, depending on the premise conditions regarding the properties of the robot, the surrounding environment, and the user specifications. In support of a navigation scheme that can cope flexibly with this variety, this paper proposes a method that formulates the problem in terms of discrete optimization problems. In the proposed method, the problem of robot position/posture estimation from map and sensor data and the problem of deciding the action at each instant are formulated as separate discrete optimization problems. In the former, our approach is an approximation of the Markov localization algorithm. The proposed method is then applied to a navigation problem in a known environment with fixed obstacles, and the effectiveness and feasibility of the method are verified by simulation results.

Motion planning of a mobile robot as a discrete optimization problem

Igarashi and Ioi (2000) proposed a solution to motion planning of a mobile robot. They formulated the problem as a discrete optimization problem at each time step. To solve the optimization problem, they used an objective function consisting of a goal term, a smoothness term and a collision term. We propose a theoretical method using reinforcement learning for adjusting weight parameters in the objective functions. However, the conventional Q-learning method cannot be applied to a non-Markov decision process, which is caused by the smoothness term. Thus, we applied Williamss (1992) learning algorithm, episodic REINFORCE, to derive a learning rule for the weight parameters. This maximizes a value function stochastically. We verified the learning rule by some experiments.