Hisashi Handa

Robust route optimization for gritting/salting trucks: a CERCIA experience

Highway authorities in marginal winter climates are responsible for the precautionary gritting/salting of the road network in order to prevent frozen roads. For efficient and effective road maintenance, accurate road surface temperature prediction is required. However, this information is useless if an effective means of utilizing this information is unavailable. This is where gritting route optimization plays a crucial role. The decision whether to grit the road network at marginal nights is a difficult problem. The consequences of making a wrong decision are serious, as untreated roads are a major hazard. However, if grit/salt is spread when it is not actually required, there are unnecessary financial and environmental costs. The goal here is to minimize the financial and environmental costs while ensuring roads that need treatment will. In this article, a salting route optimization (SRO) system that combines evolutionary algorithms with the neXt generation Road Weather Information System (XRWIS) is introduced. The synergy of these methodologies means that salting route optimization can be done at a level previously not possible.

Dynamic salting route optimisation using evolutionary computation

On marginal winter nights, highway authorities face a difficult decision as to whether or not to salt the road network. The consequences of making a wrong decision are serious, as an untreated network is a major hazard. However, if salt is spread when it is not actually required, there are unnecessary financial and environmental consequences. In this paper, a new salting route optimisation system is proposed which combines evolutionary computation (EC) with the next generation road weather information systems (XRWIS). XRWIS is a new high resolution forecast system which predicts road surface temperature and condition across the road network over a 24 hour period. ECs are used to optimise a series of salting routes for winter gritting by considering XRWIS temperature data along with treatment vehicle and road network constraints. This synergy realises daily dynamic routing and it will yield considerable benefits for areas with a marginal ice problem

Estimation of distribution algorithms with mutation

The Estimation of Distribution Algorithms are a class of evolutionary algorithms which adopt probabilistic models to reproduce the genetic information of the next generation, instead of conventional crossover and mutation operations. In this paper, we propose new EDAs which incorporate mutation operator to conventional EDAs in order to keep the diversities in EDA populations. Empirical experiments carried out this paper confirm us the effectiveness of the proposed methods.

Solving Multi-objective Reinforcement Learning Problems by EDA-RL - Acquisition of Various Strategies

EDA-RL, Estimation of Distribution Algorithms for Reinforcement Learning Problems, have been proposed by us recently. The EDA-RL can improve policies by EDA scheme: First, select better episodes. Secondly, estimate probabilistic models, i.e., policies, and finally, interact with the environment for generating new episodes. In this paper, the EDA-RL is extended for Multi-Objective Reinforcement Learning Problems, where reward is given by several criteria. By incorporating the notions in Evolutionary Multi-Objective Optimization, the proposed method is enable to acquire various strategies by a single run.

Advanced Intelligent Paradigms in Computer Games

This book presents a sample of the most recent research concerning the application of computational intelligence techniques and internet technology in computer games. The contents include: COMMONS GAME in intelligent environment; adaptive generation of dilemma-based interactive narratives; computational intelligence in racing games; evolutionary algorithms for board game players with domain knowledge; the ChessBrain project; electronic market games; EVE s entropy; capturing player enjoyment in computer games.

Dimensionality reduction of scene and enemy information in Mario

Mario AI is one of competitions on Computational Intelligence. In the case of video games, agents have to cope with a large number of input information in order to decide their actions at every time step. We have proposed the use of Isomap, a famous Manifold Learning, to reduce the dimensionality of inputs. Especially, we have applied it into scene information. In this paper, we newly extend to enemy information, where the number of enemies is not fixed. Hence, we introduce the proximity metrics in terms of enemies. The generated low-dimensional data is used for input values of Neural Networks. That is, at every time step, transferred data by using a map from raw inputs into the low-dimensional data are presented to Neural Networks. Experimental results on Mario AI environment show the effectiveness of the proposed approach.

A NOVEL HYBRID FRAMEWORK OF COEVOLUTIONARY GA AND MACHINE LEARNING

In this paper, we will propose a novel framework of hybridization of Coevolutionary Genetic Algorithm and Machine Learning. The Coevolutionary Genetic Algorithm (CGA) which has already been proposed by Handa et al. consists of two GA populations: the first GA (H-GA) population searches for the solutions in given problems, and the second GA (P-GA) population searches for effective schemata of the H-GA. The CGA adopts the notion of commensalism, a kind of co-evolution. The new hybrid framework incorporates a schema extraction mechanism by Machine Learning techniques into the CGA. Considerable improvement in its search ability is obtained by extracting more efficient and useful schemata from the H-GA population and then by incorporating those extracted schemata into the P-GA. We will examine and compare two kinds of machine learning techniques in extracting schema information: C4.5 and CN2. Several computational simulations on multidimensional knapsack problems, constraint satisfaction problems and function optimization problems will reveal the effectiveness of the proposed methods.

Neuroevolution with manifold learning for playing Mario

Evolutionary learning of neural networks, i.e., neuroevolution, has shown to play an important role in agent constitutions. It has the robustness property for dynamic, practical problems. In the case of a large number of input neurons, however, the search space of neuroevolution becomes much larger so that it is difficult to find out better policies. In this paper, Isomap, one of the manifold learning algorithms, is employed to reduce the dimensionality of the input space. The Isomap tries to reduce the dimensionality based on manifold structures in high dimensional space and to preserve local topological relationships among data. Mario AI is used as a test bed for the proposed method. Video games such as Mario, Ms. Pac-Man, and car racing have been recognised as ideal benchmark problems for computational intelligence, where they require a variety of inputs, real-time strategy, and so on, and they provide good simulators which are capable to apply CI techniques. A large number of scenes in Mario are applied by the Isomap in order to constitute a map from scene information to low dimensional data. Experimental results on the Mario AI show the effectiveness of the proposed method.

Fitness function for finding out robust solutions on time-varying functions

Evolutionary Computations in dynamic/uncertain environments have attracted much attention. Studies regarding this research subjects can be classified into four categories: Noise, Robustness, Fitness approximation, and Time-Varying function. In research on Time-Varying function, the tracking property over changes of fitness landscape has been broadly and deeply researched so far. In this paper, instead of tracking new peaks, robust solution to Time-Varying functions is introduced. Moreover, two weighted fitness functions, Exponential Weight and Linear Weight, are proposed. Experiments on modified Brankes benchmark problems on Time-Varying function reveal the effectiveness of the weighted approaches.

Evolutionary Computation on Multitask Reinforcement Learning Problems

Recently, Multitask learning, which can cope with several tasks, has attracted much attention. Multitask Reinforcement Learning introduced by Tanaka et al is a problem class where number of problem instances of Markov Decision Processes sampled from the same probability distributions is sequentially given to reinforcement learning agents. The purpose of solving this problem is to realize adaptive agents for newly given environments by using knowledge acquired from past experience. Evolutionary Algorithms are often used to solve reinforcement learning problems if problem classes are quite different with Markov Decision Processes or state-action space is quite huge. From the viewpoint of Evolutionary Algorithms studies, the Multitask Reinforcement Learning problems are regarded as dynamic problems whose fitness landscape has changed temporally. In this paper, a memory-based Evolutionary Programming which is suitable for Multitask Reinforcement Learning problems is proposed.