Vassilis P. Plagianakos

Enhancing Differential Evolution Utilizing Proximity-Based Mutation Operators

Differential evolution is a very popular optimization algorithm and considerable research has been devoted to the development of efficient search operators. Motivated by the different manner in which various search operators behave, we propose a novel framework based on the proximity characteristics among the individual solutions as they evolve. Our framework incorporates information of neighboring individuals, in an attempt to efficiently guide the evolution of the population toward the global optimum, without sacrificing the search capabilities of the algorithm. More specifically, the random selection of parents during mutation is modified, by assigning to each individual a probability of selection that is inversely proportional to its distance from the mutated individual. The proposed framework can be applied to any mutation strategy with minimal changes. In this paper, we incorporate this framework in the original differential evolution algorithm, as well as other recently proposed differential evolution variants. Through an extensive experimental study, we show that the proposed framework results in enhanced performance for the majority of the benchmark problems studied.

Evolving cognitive and social experience in Particle Swarm Optimization through Differential Evolution: A hybrid approach

In recent years, the Particle Swarm Optimization has rapidly gained increasing popularity and many variants and hybrid approaches have been proposed to improve it. In this paper, motivated by the behavior and the spatial characteristics of the social and cognitive experience of each particle in the swarm, we develop a hybrid framework that combines the Particle Swarm Optimization and the Differential Evolution algorithm. Particle Swarm Optimization has the tendency to distribute the best personal positions of the swarm particles near to the vicinity of problems optima. In an attempt to efficiently guide the evolution and enhance the convergence, we evolve the personal experience or memory of the particles with the Differential Evolution algorithm, without destroying the search capabilities of the algorithm. The proposed framework can be applied to any Particle Swarm Optimization algorithm with minimal effort. To evaluate the performance and highlight the different aspects of the proposed framework, we initially incorporate six classic Differential Evolution mutation strategies in the canonical Particle Swarm Optimization, while afterwards we employ five state-of-the-art Particle Swarm Optimization variants and four popular Differential Evolution algorithms. Extensive experimental results on 25 high dimensional multimodal benchmark functions along with the corresponding statistical analysis, suggest that the hybrid variants are very promising and significantly improve the original algorithms in the majority of the studied cases.

Finding multiple global optima exploiting differential evolution's niching capability

Handling multimodal functions is a very important and challenging task in evolutionary computation community, since most of the real-world applications exhibit highly multi-modal landscapes. Motivated by the dynamics and the proximity characteristics of Differential Evolutions mutation strategies tending to distribute the individuals of the population to the vicinity of the problems minima, we introduce two new Differential Evolution mutation strategies. The new mutation strategies incorporate spatial information about the neighborhood of each potential solution and exhibit a niching formation, without incorporating any additional parameter. Experimental results on eight well known multimodal functions and comparisons with some state-of-the-art algorithms indicate that the proposed mutation strategies are competitive and very promising, since they are able to reliably locate and maintain many global optima throughout the evolution process.

Training neural networks with threshold activation functions and constrained integer weights

Evolutionary neural network training algorithms are presented. These algorithms are applied to train neural networks with weight values confined to a narrow band of integers. We constrain the weights and biases in the range [-2/sup k+1/+1, 2/sup k-1/-1], for k=3, 4, 5, thus they can be represented by just k bits. Such neural networks are better suited for hardware implementation than the real weight ones. Mathematical operations that are easy to implement in software might often be very burdensome in the hardware and therefore more costly. Hardware-friendly algorithms are essential to ensure the functionality and cost effectiveness of the hardware implementation. To this end, in addition to the integer weights, the trained neural networks use threshold activation functions only, so hardware implementation is even easier. These algorithms have been designed keeping in mind that the resulting integer weights require less bits to be stored and the digital arithmetic operations between them are easier to be implemented in hardware. Obviously, if the network is trained in a constrained weight space, smaller weights are found and less memory is required. On the other hand, as we have found here, the network training procedure can be more effective and efficient when larger weights are allowed. Thus, for a given application a trade off between effectiveness and memory consumption has to be considered. Our intention is to present results of evolutionary algorithms on this difficult task. Based on the application of the proposed class of methods on classical neural network benchmarks, our experience is that these methods are effective and reliable.

Hardware-friendly Higher-Order Neural Network Training using Distributed Evolutionary Algorithms

In this paper, we study the class of Higher-Order Neural Networks and especially the Pi-Sigma Networks. The performance of Pi-Sigma Networks is evaluated through several well known Neural Network Training benchmarks. In the experiments reported here, Distributed Evolutionary Algorithms are implemented for Pi-Sigma neural networks training. More specifically the distributed versions of the Differential Evolution and the Particle Swarm Optimization algorithms have been employed. To this end, each processor is assigned a subpopulation of potential solutions. The subpopulations are independently evolved in parallel and occasional migration is employed to allow cooperation between them. The proposed approach is applied to train Pi-Sigma Networks using threshold activation functions. Moreover, the weights and biases were confined to a narrow band of integers, constrained in the range [-32,32]. Thus, the trained Pi-Sigma neural networks can be represented by using 6bits. Such networks are better suited than the real weight ones for hardware implementation and to some extend are immune to low amplitude noise that possibly contaminates the training data. Experimental results suggest that the proposed training process is fast, stable and reliable and the distributed trained Pi-Sigma Networks exhibited good generalization capabilities.

Clustering in evolutionary algorithms to efficiently compute simultaneously local and global minima

In this paper a new clustering operator for evolutionary algorithms is proposed. The operator incorporates the unsupervised k-windows clustering algorithm, utilizing already computed pieces of information regarding the search space in an attempt to discover regions containing groups of individuals located close to different minimizers. Consequently, the search is confined inside these regions and a large number of global and local minima of the objective function can be efficiently computed. Extensive experiments shown that the proposed approach is effective and reliable, and greatly accelerates the convergence speed of the considered algorithms.

Deterministic nonmonotone strategies for effective training of multilayer perceptrons

We present deterministic nonmonotone learning strategies for multilayer perceptrons (MLPs), i.e., deterministic training algorithms in which error function values are allowed to increase at some epochs. To this end, we argue that the current error function value must satisfy a nonmonotone criterion with respect to the maximum error function value of the M previous epochs, and we propose a subprocedure to dynamically compute M. The nonmonotone strategy can be incorporated in any batch training algorithm and provides fast, stable, and reliable learning. Experimental results in different classes of problems show that this approach improves the convergence speed and success percentage of first-order training algorithms and alleviates the need for fine-tuning problem-depended heuristic parameters.

Evolving cognitive and social experience in Particle Swarm Optimization through Differential Evolution

In recent years, the Particle Swarm Optimization has rapidly gained increasing popularity and many variants and hybrid approaches have been proposed to improve it. Motivated by the behavior and the proximity characteristics of the social and cognitive experience of each particle in the swarm, we develop a hybrid approach that combines the Particle Swarm Optimization and the Differential Evolution algorithm. Particle Swarm Optimization has the tendency to distribute the best personal positions of the swarm near to the vicinity of problems optima. In an attempt to efficiently guide the evolution and enhance the convergence, we evolve the personal experience of the swarm with the Differential Evolution algorithm. Extensive experimental results on twelve high dimensional multimodal benchmark functions indicate that the hybrid variants are very promising and improve the original algorithm.

Enhancing principal direction divisive clustering

While data clustering has a long history and a large amount of research has been devoted to the development of numerous clustering techniques, significant challenges still remain. One of the most important of them is associated with high data dimensionality. A particular class of clustering algorithms has been very successful in dealing with such datasets, utilising information driven by the principal component analysis. In this work, we try to deepen our understanding on what can be achieved by this kind of approaches. We attempt to theoretically discover the relationship between true clusters in the data and the distribution of their projection onto the principal components. Based on such findings, we propose appropriate criteria for the various steps involved in hierarchical divisive clustering and develop compilations of them into new algorithms. The proposed algorithms require minimal user-defined parameters and have the desirable feature of being able to provide approximations for the number of clusters present in the data. The experimental results indicate that the proposed techniques are effective in simulated as well as real data scenarios.

Statistical data mining of streaming motion data for activity and fall recognition in assistive environments

The analysis of human motion data is interesting in the context of activity recognition or emergency event detection, especially in the case of elderly or disabled people living independently in their homes. Several techniques have been proposed for identifying such distress situations using either motion, audio and video sensors on the monitored subject (wearable sensors) or devices installed at the surrounding environment. Visual data captured from the users environment, using overhead cameras along with motion data, which are collected from accelerometers on the subjects body, can be fed to activity detection systems that can detect emergency situations like falls and injuries. The output of these sensors is data streams that require real time recognition, especially in such emergency situations. In this paper, we study motion and activity related streaming data and we propose classification schemes using traditional classification approaches. However, such approaches may not be always applicable for immediate alarm triggering and fall prevention or when CPU power and memory resources are limited (e.g.running the detection algorithm on a mobile device such as smartphones). To this end, we also propose a statistical mining methodology that may be used for real time motion data processing. The paper includes details of the stream data analysis methodology incorporated in the activity recognition and fall detection system along with an initial evaluation of the achieved accuracy in detecting falls. The results are promising and indicate that using the proposed methodology real time fall detection is feasible.