Sandeep Paul

Subsethood-product fuzzy neural inference system (SuPFuNIS)

A new subsethood-product fuzzy neural inference system (SuPFuNIS) is presented in this paper. It has the flexibility to handle both numeric and linguistic inputs simultaneously. Numeric inputs are fuzzified by input nodes which act as tunable feature fuzzifiers. Rule based knowledge is easily translated directly into a network architecture. Connections in the network are represented by Gaussian fuzzy sets. The novelty of the model lies in a combination of tunable input feature fuzzifiers; fuzzy mutual subsethood-based activation spread in the network; use of the product operator to compute the extent of firing of a rule; and a volume-defuzzification process to produce a numeric output. Supervised gradient descent is employed to train the centers and spreads of individual fuzzy connections. A subsethood-based method for rule generation from the trained network is also suggested. SuPFuNIS can be applied in a variety of application domains. The model has been tested on Mackey-Glass time series prediction, Iris data classification, Hepatitis medical diagnosis, and function approximation benchmark problems. We also use a standard truck backer-upper control problem to demonstrate how expert knowledge can be used to augment the network. The performance of SuPFuNIS compares excellently with other various existing models.

GPU based parallel cooperative Particle Swarm Optimization using C-CUDA: A case study

The applications requiring massive computations may get benefit from the Graphics Processing Units (GPUs) with Compute Unified Device Architecture (CUDA) by reducing the execution time. Since the introduction of CUDA, applications from different areas have been benefited. Evolutionary algorithms are one such potential area where CUDA implementation proves to be beneficial not only in terms of the speedups obtained but also the improvement in convergence time. In this paper we present a detailed study of parallel implementation of one of the existing variants of Particle Swarm Optimization which is Cooperative Particle Swarm Optimization (CPSO). We also present a comparative study on CPSO implemented in C and C-CUDA. The algorithm was tested on a set of standard benchmark optimization functions. In this process, some interesting results related to the speedup and improvements in the time in convergence were obtained. The differences in randomizing procedures used in CUDA seem to contribute towards the diversity in population leading to better solution in contrast with the serial implementation. It also provides motivation for further research on neural network architecture and weight optimization using CUDA implementation. The results obtained in this paper therefore re-emphasize the utility of CUDA based implementation for complex and computationally intensive applications.

Automatic simultaneous architecture and parameter search in fuzzy neural network learning using novel variable length crossover differential evolution

The automatic simultaneous search of structure and parameters in fuzzy-neural networks is a pressing research problem. This paper introduces a novel and powerful variable-length-crossover differential evolution algorithm, vlX-DE, which is applied to ASuPFuNIS fuzzy-neural model learning, and permits simultaneous evolution of mixed-length populations of strings representing ASuPFuNIS network instances in different rules spaces. As hybrid populations of strings evolve using vlX-DE, the population gradually converges to a single rule space after which parameter search within that space proceeds till the end of the algorithm run. Search can be directed to stress either rule node economy or minimize the sum-square-error, or trade-off between these two. Tests on three benchmark problems-iris classification, CHEM classification, and Narazaki-function approximation-clearly highlight the effectiveness of the algorithm in being able to perform this simultaneous search.

A review on advances in deep learning

Over the years conventional neural networks has shown state-of-art performance on many problems. However, their performance on recognition system is still not widely accepted in the machine learning community because these networks are unable to handle selectivity-invariance dilemma and also suffer from the problem of vanishing gradients. Some of these issues have been addressed by deep learning. Deep learning approaches attempt to disentangle intricate aspects of input by creating multiple levels of representation. These approaches have shown astonishing results in problem domains like recognition system, natural language processing, medical sciences, and in many other fields. The paper presents an overview of different deep learning approaches in a nutshell and also highlights some limitations which are restricting performance of deep neural networks in order to handle more realistic problems.

Parallel Interval Type-2 Subsethood Neural Fuzzy Inference System

A Subsethood based interval type-2 fuzzy neural evolutionary inference system.Model is implemented on a parallel platform, it learns using differential evolution.This model is hybrid of type-1 and type-2 fuzzy sets.Works excellent on function approx., time series prediction, control applications.This model handles uncertainty with lesser number of trainable parameters. Neuro-fuzzy models are being increasingly employed in the domains like weather forecasting, stock market prediction, computational finance, control, planning, physics, economics and management, to name a few. These models enable one to predict system behavior in a more human-like manner than their crisp counterparts. In the present work, an interval type-2 neuro-fuzzy evolutionary subsethood based model has been proposed for its use in finding solutions to some well-known problems reported in the literature such as regression analysis, data mining and research problems relevant to expert and intelligent systems. A novel subsethood based interval type-2 fuzzy inference system, named as Interval Type-2 Subsethood Neural Fuzzy Inference System (IT2SuNFIS) is proposed in the present work. Mathematical modeling and empirical studies clearly bring out the efficacy of this model in a wide variety of practical problems such as Truck backer-upper control, Mackey-Glass time-series prediction, Narazaki-Ralescu and bell function approximation. The simulation results demonstrate intelligent decision making capability of the proposed system based on the available data. The major contribution of this work lies in identifying subsethood as an efficient measure for finding correlation in interval type-2 fuzzy sets and applying this concept to a wide variety of problems pertaining to expert and intelligent systems. Subsethood between two type-2 fuzzy sets is different from the commonly used sup-star methods. In the proposed model, this measure assists in providing better contrast between dissimilar objects. This method, coupled with the uncertainty handling capacity of type-2 fuzzy logic system, results in better trainability and improved performance of the system. The integration of subsethood with type-2 fuzzy logic system is a novel idea with several advantages, which is reported for the first time in this paper.

Novel hybrid compact genetic algorithm for simultaneous structure and parameter learning of neural networks

The automatic simultaneous selection of structure and parameters of an artificial neural networks is an important area of research. Although many variants of evolutionary algorithms (EA) have been successfully applied to this problem, their demanding memory requirements have restricted their application to real world problems, especially embedded applications with memory constraints. In this paper, structure and parameter learning of a neural network using a novel hybrid compact genetic algorithm (HCGA) is proposed. In the HCGA, each string combines real and binary segments together. For a feedforward neural network, the real segment encodes it weights, while the binary segment encodes the presence/absence of a connection of the network. The proposed hybrid compact genetic algorithm (HCGA) has several advantages: low computational cost, controllable weight regularization leading to automatic architecture discovery. The HCGA is tested on two benchmark problems of Ripleys synthetic 2-class problem and Mackey glass time series prediction problem. Experimental results show that the proposed algorithm exhibits good performance with low computation cost and controllable network structure.

Fuzzy neural inference system using mutual subsethood products with applications in medical diagnosis and control

Presents medical diagnosis and control applications of a fuzzy neural inference system that admits both numeric as well as linguistic inputs. Numeric inputs are fuzzified prior to their application to the network; linguistic inputs are presented directly. The network architecture directly embeds fuzzy if-then rules, and connections represent antecedent and consequent fuzzy sets. The novelty of the model lies in its mutual subsethood based activation spread to rule nodes which compute fuzzy inner products. Outputs are computed using volume defuzzification, and gradient descent learning is used to train the network. The model is very economical in terms of the number of rules required to solve difficult problems. Simulation results on two benchmark problems-the Hepatitis data set and the truck backer-upper problem-show that the subsethood based model performs excellently for both applications.

Neuro-fuzzy inference system (ASuPFuNIS) model for intervention time series prediction of electricity prices

This paper presents an approach to time series prediction based on Asymmetric Subsethood-Product Fuzzy Neural Inference System (ASuPFuNIS). The standard time series techniques have standard averaging where a fixed weight is added to the past values. In this paper we present a novel neuro-fuzzy inference system based on asymmetric subsethood with intervention based transfer function based time series model for accurate prediction of time series. The design of the model is described, and the scheme is evaluated by application to real-world problem of cost of electricity prices over a period of seven year in Ontario, Canada. We also study the various statistical properties of the data.

Evolutionary Subsethood Product Fuzzy Neural Network

This paper employs a simple genetic algorithm (GA) to search for an optimal set of parameters for a novel subsethood product fuzzy neural network introduced elsewhere, and to demonstrate the pattern classification capabilities of the network. The search problem has been formulated as an optimization problem with an objective to maximize the number of correctly classified patterns. The performance of the network, with GA evolved parameters, is evaluated by computer simulations on Ripleys synthetic two class data. The network performed excellently by being at par with the Bayes optimal classifier, giving the best possible error rate of 8%. The evolutionary subsethood product network outperformed all other models with just two rules.

Simultaneous Structure and Parameter Learning of Convolutional Neural Network

This paper provides a solution to select a suitable architecture of convolutional neural network (CNN). A hybrid evolutionary gradient descent (HyEGD) approach is proposed to automatically evolve the architecture of CNN. The evolution of the structure is done using compact genetic algorithm (cGA) by optimizing the number of filters in each layer, and simultaneously, the associated weight parameters are tuned by stochastic gradient descent (SGD). This brings forth an effective way to search the solution space seamlessly integrating both exploration, spearheaded by cGA, and the exploitation, naturally done by SGD. Moreover, using HyEGD approach, the user specified architecture can also be evolved trading-off between two objectives—network performance on one side and network size on the other side. Experiments to illustrate the salient features of the HyEGD approach are performed on two benchmark problems: COIL-20 dataset and MNIST dataset. The results clearly highlight the powerful capability of generating architectures based on the required performance and size of network.