Nobuhiko Kondo

Multi-objective structure selection for radial basis function networks based on genetic algorithm

Radial basis function (RBF) network is well known as a good performance approach to nonlinear system modeling. Though structure selection of RBF network is an important issue, the framework of this problem has not been established. In this paper, we propose multiobjective structure selection method for RBF networks based on MOGA (multiobjective genetic algorithm). The structure of RBF networks is encoded to the chromosomes in GA, then evolved toward to Pareto-optimum for multiobjective functions concerned with model accuracy and complexity. Some numerical simulation results indicate the applicability of the proposed approach.

Pattern Classification by Evolutionary RBF Networks Ensemble Based on Multi-objective Optimization

In this paper, evolutionary multi-objective selection method of RBF networks structure and its application to the ensemble learning is considered. The candidates of RBF network structure are encoded into the chromosomes in GAs. Then, they evolve toward Pareto-optimal front defined by several objective functions concerning with model accuracy, model complexity and model smoothness. RBF network ensemble is constructed of the obtained Pareto-optimal models since such models are diverse. This method is applied to the pattern classification problem. Experiments on the benchmark problem demonstrate that the proposed method has comparable generalization ability to conventional ensemble methods.

Nonlinear Dynamic System Identification Based on Multiobjectively Selected RBF Networks

In this paper, nonlinear dynamic system identification by using multiobjectively selected RBF network is considered. RBF networks are widely used as a model structure for nonlinear systems. The determination of its structure that is the number of basis functions is prior important step in system identification, and the tradeoff between model complexity and accuracy exists in this problem. By using multiobjective evolutionary algorithms, the candidates of the RBF network structure are obtained in the sense of Pareto optimality. We discuss an application to system identification by using such RBF networks having Pareto optimal structures. Some numerical simulations for nonlinear dynamic systems are carried out to show the applicability of the proposed approach.

Search Performance Improvement for PSO in High Dimensional Space

Particle swarm optimisation (PSO) was developed by Kennedy and Eberhart in 1995 (Kennedy & Eberhart, 1995) inspired by the collective behaviour of natural birds or fish. PSO is a stochastic optimisation technique that uses a behaviour of population composed by many search points called particle. In spite of easy implementation in computer algorithms, it is well known as a powerful numerical optimizer. In the typical PSO algorithms, a set of particles searches the optimal solution in the problem space efficiently, by sharing the common attractor called global best. There are many modified versions of PSO by improving convergence property to a certain problem. While, a standard PSO is defined by Bratton and Kennedy (Bratton & Kennedy, 2007) to give a real standard for PSO studies. PSO seems as one of the evolutionary computations (ECs), and it has been shown that PSO is comparable to a genetic algorithm (Angeline, 1998). Thus, a lot of studies have demonstrated the effectiveness of PSO family in optimizing various continuous and discrete optimization problems. And a plenty of applications of PSO, such as the neural network training, PID controller tuning, electric system optimisation have been studied and achieved well results (Kennedy, 1997). However, PSO is often failed in searching the global optimal solution in the case of the objective function has a large number of dimensions. The reason of this phenomenon is not only existence of the local optimal solutions, the velocities of the particles sometimes lapsed into the degeneracy, so that the successive range is restricted in the sub-plain of the whole search hyper-plain. The sub-plane that is defined by finite number of particle velocities is a partial space in the whole search space. The issue of local optima in PSO has been studied and proposed several modifications on the basic particle driven equation (Parsopoulos et al., 2001; Hendtlass, 2005; Liang et al., 2006). There used a kind of adaptation technique or randomized method (e.g. mutation in evolutionary computations) to keep particles velocities or to accelerate them. Although such improvements work well and have ability to avoid fall in the local optima, the problem of early convergence by the degeneracy of some dimensions is still remaining, even if there are no local optima. Hence the PSO algorithm does not always work well for the high-dimensional function.

Pattern Classification via Multi-objective Evolutionary RBF Networks Ensemble

This paper considers a pattern classification by the ensemble of evolutionary RBF networks. Mathematical models generally have a dilemma about model complexity, so the structure determination of RBF network can be considered as the multi-objective optimization problem concerning with accuracy and complexity of the model. The set of RBF networks are obtained by multi-objective evolutionary computation and then RBF network ensemble is constructed of all or some RBF networks at the final generation. Some experiments on the benchmark problem of the pattern classification demonstrate that the RBF network ensemble has comparable generalization ability to conventional ensemble methods

Topology-based personal selection in multi-objective Particle Swarm Optimization

Particle Swarm Optimization (PSO) is a stochastic multi-point search algorithm. It was inspired by the social behavior observed in nature, such as flocks of birds and schools of fish. In recent years, multi-objective optimization by using PSO is receiving much attention. There are two difference steps from standard single objective PSO; 1) the use of archives to reserve Pareto optimal candidates, and 2) the selection of appropriate guides for multi-objective optimization. Topology is often used for standard PSO to make appropriate balance between exploration and exploitation. However, the use of topology in multi-objective PSO is not well studied. From this viewpoint, we propose a PSO model that introduces a topology-based guide selection scheme for multi-objective optimization, in this paper. The numerical simulation results show that the proposed guide selection method is effective in multi-objective optimization benchmark problems.

Early Detection of At-Risk Students Using Machine Learning Based on LMS Log Data

Analytics in education has been received much attention over the past decade. It is necessary to maintain high retention rate in any institutions of higher education, therefore several attempts on the application of analytics have been done for this problem. To detect students at high drop-out risk early and intervene them effectively, utilizing the educational big data can be useful. In this paper, an automatic detection method of academically at-risk students by using log data of learning management systems is considered. Some well-known machine learning methods are used to build a predictive model of student performance evaluated by GPA. By using actual data set, we investigate an availability of the proposed method and discuss its ability to early detection of off-task behavior. The experimental results indicated that some characteristics of behavior about learning which affect the learning outcomes can be detected with only the online log data. Furthermore, comparative importance of explanatory variables obtained by the approach would help to estimate which variable affects comparatively to the learning outcome and it can be used in institutional research.

Nonlinear System Modeling by Hybrid Genetic Programming

Abstract Genetic Programming (GP) is a useful tool of nonlinear model building, however a simple use of GP often fails in numeric optimization since GP hangs on random number sampling in searching appropriate constant parameters in individual representing each model candidate. From this viewpoint a hybrid GP based nonlinear system identification method is proposed in this paper. We introduce a simple numerical optimization inspired by Particle swarm in GP operation to improve numeric optimization ability. Then, this hybridization is applied to nonlinear system identification by using GP. The applicability of the proposed method is shown by the results of some numerical experiments.