Keiji Tatsumi

A chaotic particle swarm optimization exploiting a virtual quartic objective function based on the personal and global best solutions

The particle swarm optimization method (PSO) is one of population-based optimization techniques for global optimization, where a number of candidate solutions called particles simultaneously move toward the tentative solutions found by particles so far, which are called the personal and global bests, respectively. Since, in the PSO, the exploration ability is important to find a desirable solution, various kinds of methods have been investigated to improve it. In this paper, we propose a PSO with a new chaotic system derived from the steepest descent method for a virtual quartic objective function with perturbations having its global minima at the personal and global bests, where elements of each particles position are updated by the proposed chaotic system or the standard update formula. Thus, the proposed PSO can search for solutions around the personal and global bests intensively without being trapped at any local minimum due to the chaoticness. Moreover, we show approximately the sufficient condition of parameter values of the proposed system under which the system is chaotic. Through computational experiments, we verify the performance of the proposed PSO by applying it to some global optimization problems.

Multiobjective multiclass soft-margin support vector machine maximizing pair-wise interclass margins

The all together model is one of the support vector machine (SVM) for multiclass classification by using a piece-wise linear function. As a novel all together model, we already proposed a hard-margin multiobjective SVM model for piecewise linearly separable data, which maximizes all of the geometric margins simultaneously for the generalization ability. In addition, we derived a single-objective convex problem whose optimal solution is weakly Pareto optimal for the proposed SVM. However, in the real-world classification problem the data are often piecewise linearly inseparable. Therefore, in this paper we extend the hard-margin SVM for the data by introducing penalty functions for the margin slack variables based on the geometric distances between outliers and the support hyperplanes, and incorporating those functions into the objective functions. Moreover, we derive a single-objective second-order cone programming (SOCP) problem, and show that its optimal solution is weakly Pareto optimal for the proposed soft-margin SVM. Furthermore through numerical experiments we verify that the SOCP model maximizes the geometric margins in the sense of multiobjective optimization.

Particle swarm optimization with stochastic selection of perturbation-based chaotic updating system

In this paper, we consider the particle swarm optimization (PSO). In particular, we focus on an improved PSO called the CPSO-VQO, which uses a perturbation-based chaotic system and a threshold-based method of selecting from the standard and chaotic updating systems for each particle on the basis of the difference vector between its pbest and the gbest. Although it was reported that the CPSO-VQO performs well, it is not easy to select an amplitude of the perturbation and a threshold appropriately for an effective search. This is because the bifurcation structure of the chaotic system depends on the difference vector, and the difference vector varies widely between different stages of the search and between different problems.Therefore, we improve the CPSO-VQO by proposing a modified chaotic system whose bifurcation structure is irrelevant to the difference vector, and show theoretically desirable properties of the modified system. We also propose a new stochastic method that selects the updating system according to the ratio between the components of the difference vector for each particle, and restarting and acceleration techniques to develop the standard updating system used in the proposed PSO model. The proposed methods can maintain an appropriate balance between the identification and diversification aspects of the search. Moreover, we perform numerical experiments to evaluate the performance of the proposed PSOs: PSO-TPC, PSO-SPC, PSO-SDPC, IPSO-SPC and IPSO-SDPC. In particular, we demonstrate that the IPSO-SDPC finds high-quality solutions and is robust against variations in its parameter values.

A SUFFICIENT CONDITION FOR CHAOS IN THE GRADIENT MODEL WITH PERTURBATION METHOD FOR GLOBAL OPTIMIZATION

The chaotic system has been exploited in metaheuristic methods of solving global optimization problems having a large number of local minima. In those methods, the selection of chaotic system is significantly important to search for solutions extensively. Recently, a novel chaotic system, the gradient model with perturbation methods (GP), was proposed, which can be regarded as the steepest descent method for minimizing an objective function with additional perturbation terms, and it is reported that chaotic metaheuristic method with the GP model has a good performance of solving some benchmark problems through numerical experiments. Moreover, a sufficient condition of parameter was theoretically shown for chaoticity in a simplified GP model where the descent term for the objective function is removed from the original model. However, the shown conditions does not provide enough information to select parameter values in the GP model for metaheuristic methods. Therefore, in this paper, we theoretically derive a sufficient condition under which the original GP model is chaotic, which can be usefully exploited for an appropriate selection of parameter values. In addition, we examine the derived sufficient condition by calculating the Lyapunov exponents of the GP model, and analyze its bifurcation structure through numerical experiments.

Restarting multi-type particle swarm optimization using an adaptive selection of particle type

The particle swarm optimization method (PSO) is one of popular metaheuristic methods for global optimization problems. Although the PSO is simple and shows a good performance of finding a good solution, it is reported that almost all particles sometimes converge to an undesirable local minimum for some problems. Thus, many kinds of improved methods have been proposed to keep the diversity of the search process. In this paper, we propose a novel multi-type swarm PSO which uses two kinds of particles and multiple swarms including either kind of particles. All particles in each swarm search for solutions independently where the exchange of information between different swarms is restricted for the extensive exploration. In addition, the proposed model has the restarting system of inactive particles which initializes a trapped particle by resetting its velocity and position, and adaptively selects the kind of the particle according to which kind of particles contribute to improvement of the objective function. Furthermore, through some numerical experiments, we verify the abilities of the proposed model.

Multiobjective multiclass support vector machine based on the one-against-all method

Recently, some kinds of extensions of the binary support vector machine (SVM) to multiclass classification have been proposed. In this paper, we focus on the one-against-all and all-together methods, which finally construct the same kind of multiclass classifier. Since in the one-against-all method, binary SVMs are simply combined, the geometric margins of the multiclass classifier are not maximized. On the other hand, although the all-together method is aimed at maximizing the geometric margins for the generalization ability, it requires a large amount of computational resources because it is formulated as a large-scale optimization problem. In this paper, we propose a new model which constructs a multiclass classifier as a weighted combination of binary SVMs obtained by the one-against-all method and which maximizes the geometric margins. The proposed model can be expected to have the high generalization ability and reduce computational resources. Moreover, we show the advantage of the proposed model through numerical experiments.

Multiobjective Multiclass Soft-Margin Support Vector Machine and Its Solving Technique Based on Benson's Method

In this paper, we focus on the all together model, which is one of the support vector machine (SVM) using a piece-wise linear function for multiclass classification. We already proposed a multiobjective hard-margin SVM model as a new all together model for piecewise linearly separable data, which maximizes all of the geometric margins simultaneously for the generalization ability. In addition, we derived a single-objective convex problem and showed that a Pareto optimal solution for the proposed multiobjective SVM is obtained by solving single-objective problems. However, in the real-world classification problem the data are often piecewise linearly inseparable. Therefore, in this paper we extend the hard-margin SVM for the data by using penalty functions for the margin slack variables between outliers and the corresponding discriminant hyperplane. Those functions are incorporated into the objective functions. Moreover, we derive a single-objective second-order cone programming (SOCP) problem based on Bensons method and some techniques, and show that a Pareto optimal solution for the proposed soft-margin SVM is obtained by solving the SOCP iteratively. Furthermore through numerical experiments we verify that the proposed iterative method maximizes the geometric margins and constructs a classifier with a high generalization ability.

A perturbation based chaotic particle swarm optimization using multi-type swarms

In order to improve the particle swarm optimization (PSO) method, which is a popular metaheuristic method for global optimization, we already proposed a PSO exploiting a chaotic dynamical system with sinusoidal perturbations, where chaotic and standard particles search for solutions cooperatively. In this paper, we propose multi-type swarms for the chaotic PSO which has three kinds of particles, the standard, chaotic and PS particles, and two kinds of best solutions, the global best and promising solutions: The chaotic particle searches for solutions chaotically and extensively in the feasible region to update the promising solution, while the standard particle executes the detail search around the global best solution which is updated by all particles. Moreover, PS particle searches for solutions in detail around the promising solution in the same way of the standard particle to inform the promising region found by the chaotic particles to the standard particles. Through computational experiments, we verify the performance of the proposed model by applying it to some global optimization problems.

Chaotic Particle Swarm Optimization Method Exploiting Sinusoidal Perturbations

The particle swarm optimization (PSO) method is a population-based optimization technique which searches for solutions by updating simultaneously a number of candidate solutions called particles. Since, in the PSO, the exploration ability is important to find a desirable solution, various kinds of methods have been investigated to improve it. In this paper, we propose a new chaotic dynamical system for PSO having sinusoidal perturbations, in which chaotic particles can be expected to search for solutions around the two tentative solutions by using chaotic sequences. Through computational experiments, by applying them to some global optimization problems, we compare the proposed chaotic PSO with the standard one

Improved projection Hopfield network for the quadratic assignment problem

The continuous-valued Hopfield neural network (CHN) is a popular method of metaheuristics. However, it is not guaranteed that obtained solutions by the CHN are always feasible. To obtain a high-quality feasible solution, appropriate penalty parameters of CHN are required. Matsuda has shown the theoretical relationship between penalty parameters and the qualities of obtained solutions of the CHN for a traveling salesman problem (TSP). We show a similar theoretical relationship of the CHN for the quadratic assignment problem (QAP) and the limitation of the CHP. On other hand, Smith et al. proposed the projection method to obtain a high-quality feasible solution, which projects a modified solution onto two constraints, by turns. Thus, this method is not efficient and does not necessarily find a feasible solution at each iteration. Therefore, we propose a new method with the projection of a modified solution onto the entire feasible region at once. Moreover, we show convergence properties of the proposed method and the conditions of penalty parameters which guarantees that the CHN for QAP always finds the feasible solution. Finally, we verify the efficiency of the methods through numerical experiments.