Lotika Singh

GPU based parallel cooperative Particle Swarm Optimization using C-CUDA: A case study

The applications requiring massive computations may get benefit from the Graphics Processing Units (GPUs) with Compute Unified Device Architecture (CUDA) by reducing the execution time. Since the introduction of CUDA, applications from different areas have been benefited. Evolutionary algorithms are one such potential area where CUDA implementation proves to be beneficial not only in terms of the speedups obtained but also the improvement in convergence time. In this paper we present a detailed study of parallel implementation of one of the existing variants of Particle Swarm Optimization which is Cooperative Particle Swarm Optimization (CPSO). We also present a comparative study on CPSO implemented in C and C-CUDA. The algorithm was tested on a set of standard benchmark optimization functions. In this process, some interesting results related to the speedup and improvements in the time in convergence were obtained. The differences in randomizing procedures used in CUDA seem to contribute towards the diversity in population leading to better solution in contrast with the serial implementation. It also provides motivation for further research on neural network architecture and weight optimization using CUDA implementation. The results obtained in this paper therefore re-emphasize the utility of CUDA based implementation for complex and computationally intensive applications.

Automatic simultaneous architecture and parameter search in fuzzy neural network learning using novel variable length crossover differential evolution

The automatic simultaneous search of structure and parameters in fuzzy-neural networks is a pressing research problem. This paper introduces a novel and powerful variable-length-crossover differential evolution algorithm, vlX-DE, which is applied to ASuPFuNIS fuzzy-neural model learning, and permits simultaneous evolution of mixed-length populations of strings representing ASuPFuNIS network instances in different rules spaces. As hybrid populations of strings evolve using vlX-DE, the population gradually converges to a single rule space after which parameter search within that space proceeds till the end of the algorithm run. Search can be directed to stress either rule node economy or minimize the sum-square-error, or trade-off between these two. Tests on three benchmark problems-iris classification, CHEM classification, and Narazaki-function approximation-clearly highlight the effectiveness of the algorithm in being able to perform this simultaneous search.

Parallel Evolutionary Asymmetric Subsethood Product Fuzzy-Neural Inference System with Applications

This paper introduces PEASuPFuNIS, a parallel evolutionary asymmetric subsethood product fuzzy neural network as an extension of ASuPFuNIS, which is implemented using a high performance LAM/MPI cluster. EASuPFuNlS employs differential evolution learning which is parallelized using a master-slave model, and the implementation is facilitated through the use of derived data-types. Parallelization of EASuPFuNIS using DE learning leads to super-linear speedups concomitant with high performance as is shown through instrumentation using two problems: the Hang function approximation problem, and the Mackey-Glass time series prediction problem. Parallelization and ran-time speedup of the EASuPFuNIS model opens up the possibility of applying this class of models to real world problem domains which was hitherto not possible with the serial version due to the requirement of large computation time.

A review on advances in deep learning

Over the years conventional neural networks has shown state-of-art performance on many problems. However, their performance on recognition system is still not widely accepted in the machine learning community because these networks are unable to handle selectivity-invariance dilemma and also suffer from the problem of vanishing gradients. Some of these issues have been addressed by deep learning. Deep learning approaches attempt to disentangle intricate aspects of input by creating multiple levels of representation. These approaches have shown astonishing results in problem domains like recognition system, natural language processing, medical sciences, and in many other fields. The paper presents an overview of different deep learning approaches in a nutshell and also highlights some limitations which are restricting performance of deep neural networks in order to handle more realistic problems.

Parallel Evolutionary Asymmetric Subsethood Product Fuzzy-Neural Inference System: An Island Model Approach

This paper introduces an island model approach for differential evolution (DE) learning in asymmetric subsethood product fuzzy neural inference system (ASuPFuNIS). In the island model, each island executes an independent DE and maintains its own sub-population for search. The migration model scheme has been implemented here to parallelize ASuPFuNIS. The parallelization strategy presented here is compared with the master-slave approach

Parallel Interval Type-2 Subsethood Neural Fuzzy Inference System

A Subsethood based interval type-2 fuzzy neural evolutionary inference system.Model is implemented on a parallel platform, it learns using differential evolution.This model is hybrid of type-1 and type-2 fuzzy sets.Works excellent on function approx., time series prediction, control applications.This model handles uncertainty with lesser number of trainable parameters. Neuro-fuzzy models are being increasingly employed in the domains like weather forecasting, stock market prediction, computational finance, control, planning, physics, economics and management, to name a few. These models enable one to predict system behavior in a more human-like manner than their crisp counterparts. In the present work, an interval type-2 neuro-fuzzy evolutionary subsethood based model has been proposed for its use in finding solutions to some well-known problems reported in the literature such as regression analysis, data mining and research problems relevant to expert and intelligent systems. A novel subsethood based interval type-2 fuzzy inference system, named as Interval Type-2 Subsethood Neural Fuzzy Inference System (IT2SuNFIS) is proposed in the present work. Mathematical modeling and empirical studies clearly bring out the efficacy of this model in a wide variety of practical problems such as Truck backer-upper control, Mackey-Glass time-series prediction, Narazaki-Ralescu and bell function approximation. The simulation results demonstrate intelligent decision making capability of the proposed system based on the available data. The major contribution of this work lies in identifying subsethood as an efficient measure for finding correlation in interval type-2 fuzzy sets and applying this concept to a wide variety of problems pertaining to expert and intelligent systems. Subsethood between two type-2 fuzzy sets is different from the commonly used sup-star methods. In the proposed model, this measure assists in providing better contrast between dissimilar objects. This method, coupled with the uncertainty handling capacity of type-2 fuzzy logic system, results in better trainability and improved performance of the system. The integration of subsethood with type-2 fuzzy logic system is a novel idea with several advantages, which is reported for the first time in this paper.

Novel hybrid compact genetic algorithm for simultaneous structure and parameter learning of neural networks

The automatic simultaneous selection of structure and parameters of an artificial neural networks is an important area of research. Although many variants of evolutionary algorithms (EA) have been successfully applied to this problem, their demanding memory requirements have restricted their application to real world problems, especially embedded applications with memory constraints. In this paper, structure and parameter learning of a neural network using a novel hybrid compact genetic algorithm (HCGA) is proposed. In the HCGA, each string combines real and binary segments together. For a feedforward neural network, the real segment encodes it weights, while the binary segment encodes the presence/absence of a connection of the network. The proposed hybrid compact genetic algorithm (HCGA) has several advantages: low computational cost, controllable weight regularization leading to automatic architecture discovery. The HCGA is tested on two benchmark problems of Ripleys synthetic 2-class problem and Mackey glass time series prediction problem. Experimental results show that the proposed algorithm exhibits good performance with low computation cost and controllable network structure.

Dynamic fuzzy load balancing on LAM/MPI clusters with applications in parallel master-slave implementations of an evolutionary neuro-fuzzy learning system

In the context of parallel master-slave implementations of evolutionary learning in fuzzy-neural network models, a major issue that arises during runtime is how to balance the load-the number of strings assigned to a slave for evaluation during a generation-in order to achieve maximum speed up. Slave evaluation times can fluctuate drastically depending upon the local computational load on the slave (given fixed node specifications). Communication delays compound the problem of proper load assignment. In this paper we propose the design of a novel dynamic fuzzy load estimator for application to load balancing on heterogeneous LAM/MPI clusters. Using average evaluation time and communication delay feedback estimates from slaves, string assignments for evaluation to slaves are dynamically changed during runtime. Extensive tests on heterogenous clusters shows that considerable speedups can be achieved using the proposed fuzzy controller.

A novel adaptive fuzzy load balancer for heterogeneous LAM/MPI clusters applied to evolutionary learning in neuro-fuzzy systems

Load balancing in parallel master-slave implementations on heterogeneous computing clusters is a pressing research problem. Proper load balancing can lead to dramatic speedups in program run times. This paper introduces a novel adaptive fuzzy load balancer which automatically senses cluster state through measurements of node evaluation times and network delays. Measured data are collected within a time window and then clustered using fuzzy c-means clustering. The optimal number of clusters are decided using the Xie-Beni index. Rule base extraction is facilitated by reverse projection of clusters (for antecedents) and a heuristic function (for consequents). Re-clustering is triggered on outlier point detection, and re-validation of clusters is performed depending on an FCM objective function-based cluster scattering threshold. The load balancer is deployed on the master to balance the load between various slaves. The algorithm is tested extensively on an evolutionary-neuro-fuzzy network learning application and implemented in a LAM/MPI computing environment. Results clearly bring out the efficacy of employing the adaptive load balancer in heterogeneous computing environments. Speedups ranging from 42% to 89% are observed when compared to parallel implementations without the fuzzy load balancer, and up to 448% when compared to the serial implementations.

Migration based parallel differential evolution learning in Asymmetric Subsethood Product Fuzzy Neural Inference System :A simulation study

This paper presents a detailed study on the various parameters of an island model based differential evolution learning scheme in asymmetric subsethood product fuzzy neural inference system (ASuPFuNIS). The systematic experimental study of the migration size, migration interval coupled with an in depth view of the diversity on each island leads to a better understanding of the algorithm. In the course of studying the effects of parameters some significant performance changes were observed on a standard benchmark function approximation problem.