Udayan Ghose

A non-sigmoidal activation function for feedforward artificial neural networks

For a single hidden layer feedforward artificial neural network to possess the universal approximation property, it is sufficient that the hidden layer nodes activation functions are continuous non-polynomial function. It is not required that the activation function be a sigmoidal function. In this paper a simple continuous, bounded, non-constant, differentiable, non-sigmoid and non-polynomial function is proposed, for usage as the activation function at hidden layer nodes. The proposed activation function does require the computation of an exponential function, and thus is computationally less intensive as compared to either the log-sigmoid or the hyperbolic tangent function. On a set of 10 function approximation tasks we demonstrate the efficiency and efficacy of the usage of the proposed activation functions. The results obtained allow us to assert that, at least on the 10 function approximation tasks, the results demonstrate that in equal epochs of training, the networks using the proposed activation function reach deeper minima of the error functional and also generalize better in most of the cases, and statistically are as good as if not better than networks using the logistic function as the activation function at the hidden nodes.

Bi-modal derivative adaptive activation function sigmoidal feedforward artificial neural networks

Abstract In this work an adaptive mechanism for choosing the activation function is proposed and described. Four bi-modal derivative sigmoidal adaptive activation function is used as the activation function at the hidden layer of a single hidden layer sigmoidal feedforward artificial neural networks. These four bi-modal derivative activation functions are grouped as asymmetric and anti-symmetric activation functions (in groups of two each). For the purpose of comparison, the logistic function (an asymmetric function) and the function obtained by subtracting 0.5 from it (an anti-symmetric) function is also used as activation function for the hidden layer nodes’. The resilient backpropagation algorithm with improved weight-tracking (iRprop+) is used to adapt the parameter of the activation functions and also the weights and/or biases of the sigmoidal feedforward artificial neural networks. The learning tasks used to demonstrate the efficacy and efficiency of the proposed mechanism are 10 function approximation tasks and four real benchmark problems taken from the UCI machine learning repository. The obtained results demonstrate that both for asymmetric as well as anti-symmetric activation usage, the proposed/used adaptive activation functions are demonstratively as good as if not better than the sigmoidal function without any adaptive parameter when used as activation function of the hidden layer nodes.

A Linguistic Rule-Based Approach for Aspect-Level Sentiment Analysis of Movie Reviews

Aspect-level sentiment analysis refers to sentiment polarity detection from unstructured text at a fine-grained feature or aspect level. This paper presents our experimental work on aspect-level sentiment analysis of movie reviews. Movie reviews generally contain user opinion about different aspects such as acting, direction, choreography, cinematography, etc. We have devised a linguistic rule-based approach which identifies the aspects from movie reviews, locates opinion about that aspect and computes the sentiment polarity of that opinion using linguistic approaches. The system generates an aspect-level opinion summary. The experimental design is evaluated on datasets of two movies. The results achieved good accuracy and shows promise for deployment in an integrated opinion profiling system.

Effect of Activation Function Symmetry on Training of SFFANNs with RPROP Algorithm

On ten learning tasks (5 function approximation and 5 real life regression problems), we compare the effciency and efficacy of using asymmetric or anti-symmetric activation functions in sigmoidal feedforward artificial neural network training and usage. The result obtained in the experiment allows us to conclude that for networks trained using the improved variant of the resilient backpropagation algorithm, the usage of asymmetric activation functions like the logistic or the log-sigmoid function should be preferred as compared to anti-symmetric function such that the two functions have the same derivative.

Attribute Reduction Method Using the Combination of Entropy and Fuzzy Entropy

The enormous size datasets are being used in various fields such as administration, engineering, management and so on. For information retreival from these datasets more time is being consumed. Fewer attribute datasets takes lesser time for computation, and are more understandable and intelligible. Attribute reduction is a tool for feature selection as it transforms data into knowledge. A new method using the combination of entropy and fuzzy entropy is proposed for removal of redundancy and irrelevant attributes which results in reducing the dataset size. The functioning of the proposed method is examined on standard datasets such as Sonar, Spambase and Tick-tack-Toe. Experimental results performed on various datasets show that proposed method gives significant improvement in attribute reduction. In this work, nearest neighbor classifier is used to examine the classification accuracy on original and reduced dataset.

Rule based information leveraging from business invoices

Invoices are interchanged between business organizations on a day-to-day basis, and they all contain the similar kind of information i.e. What is the name of issuing company? To whom is it issued? What is the amount of the invoice? What is the mode of payment? Capturing and structuring this information can give floor to critical supply chain and cash flow questions and help business analysts make better decisions. In order to represent data with irregular or hidden structure, semi structured data allows a “schema-less” description format in which the data is loosely constrained than in usual database system. In this paper annotation is used on the basis of some rules to add more structure to business invoices in order to simplify and standardize the storage and retrieval of business information.

A comparative study of classification algorithms for forecasting rainfall

India is an agricultural country which largely depends on monsoon for irrigation purpose. A large amount of water is consumed for industrial production, crop yield and domestic use. Rainfall forecasting is thus very important and necessary for growth of the country. Weather factors including mean temperature, dew point temperature, humidity, pressure of sea and speed of wind and have been used to forecasts the rainfall. The dataset of 2245 samples of New Delhi from June to September (rainfall period) from 1996 to 2014 has been collected from a website named Weather Underground. The training dataset is used to train the classifier using Classification and Regression Tree algorithm, Naive Bayes approach, K nearest Neighbour and 5-10-1 Pattern Recognition Neural Network and its accuracy is tested on a test dataset. Pattern Recognition networks has given 82.1% accurate results, KNN with 80.7% correct forecasts ranks second, Classification and Regression Tree(CART) gives 80.3% while Naive Bayes provides 78.9% correctly classified samples.

On the feasibility of solving regression learning tasks with FFANN using non-sigmoidal activation functions

In this paper, parametrized non-sigmoidal, continuous and bounded function(s) are proposed as the activation function at the hidden nodes of a feedforward artificial neural networks (FFANN). On a set of 5 regression (benchmark) tasks that correspond to real-life learning problems, the effect of the usage of the parametrized function as the activation function at the hidden layer nodes, on the efficiency and efficacy of training the FFANN is studied. It is observed that on the given set of problems, one of the parameterized activation function (with a particular parameter value), gives statistically meaningful results (lower minima of the error functional during training) as compared to the standard log-sigmoid activation function in 4 cases while in the fifth problem, the two activations are found to be statistically equivalent.