Ranadhir Ghosh

A HIERARCHICAL METHOD FOR FINDING OPTIMAL ARCHITECTURE AND WEIGHTS USING EVOLUTIONARY LEAST SQUARE BASED LEARNING

In this paper, we present a novel approach of implementing a combination methodology to find appropriate neural network architecture and weights using an evolutionary least square based algorithm (GALS).1 This paper focuses on aspects such as the heuristics of updating weights using an evolutionary least square based algorithm, finding the number of hidden neurons for a two layer feed forward neural network, the stopping criterion for the algorithm and finally some comparisons of the results with other existing methods for searching optimal or near optimal solution in the multidimensional complex search space comprising the architecture and the weight variables. We explain how the weight updating algorithm using evolutionary least square based approach can be combined with the growing architecture model to find the optimum number of hidden neurons. We also discuss the issues of finding a probabilistic solution space as a starting point for the least square method and address the problems involving fitness breaking. We apply the proposed approach to XOR problem, 10 bit odd parity problem and many real-world benchmark data sets such as handwriting data set from CEDAR, breast cancer and heart disease data sets from UCI ML repository. The comparative results based on classification accuracy and the time complexity are discussed.

A feature extraction technique for online handwriting recognition

The paper presents a feature extraction technique for online handwriting recognition. The technique incorporates many characteristics of handwritten characters based on structural, directional and zoning information and combines them to create a single global feature vector. The technique is independent to character size and it can extract features from the raw data without resizing. Using the proposed technique and a neural network based classifier, many experiments were conducted on UNIPEN benchmark database. The recognition rates are 98.2% for digits, 91.2% for uppercase and 91.4% for lowercase.

A novel evolutionary neural learning algorithm

We present a novel genetic algorithm and least square (GALS) based hybrid learning approach for the training of an artificial neural network (ANN). The approach combines evolutionary algorithms with matrix solution methods such as Gram-Schmidt, SVD, etc., to adjust weights for hidden and output layers. Our hybrid method (GALS) incorporates the evolutionary algorithm (EA) in the first layer and the least square method (LS) in the second layer of the ANN. In the proposed approach, a two-layer network is considered, the hidden layer weights are evolved using an evolutionary algorithm and the output layer weights are calculated using a linear least square method. When a certain number of generation or error goals in terms of RMS error is reached, the training is stopped. We start training with a small number of hidden neurons and then the number is increased gradually in an incremental process. The proposed algorithm was implemented and many experiments were conducted on benchmark data sets such as XOR, 10-bit odd parity, handwritten segmented characters recognition, breast cancer diagnosis and heart disease data. The experimental results showed very promising results when compared with other existing evolutionary and error back propagation (EBP) algorithms in classification rate and time complexity.

A variable initialization approach to the EM algorithm for better estimation of the parameters of hidden markov model based acoustic modeling of speech signals

The traditional method for estimation of the parameters of Hidden Markov Model (HMM) based acoustic modeling of speech uses the Expectation-Maximization (EM) algorithm. The EM algorithm is sensitive to initial values of HMM parameters and is likely to terminate at a local maximum of likelihood function resulting in non-optimized estimation for HMM and lower recognition accuracy. In this paper, to obtain better estimation for HMM and higher recognition accuracy, several candidate HMMs are created by applying EM on multiple initial models. The best HMM is chosen from the candidate HMMs which has highest value for likelihood function. Initial models are created by varying maximum frame number in the segmentation step of HMM initialization process. A binary search is applied while creating the initial models. The proposed method has been tested on TIMIT database. Experimental results show that our approach obtains improved values for likelihood function and improved recognition accuracy.

A hybrid question answering schema using encapsulated semantics in lexical resources

Question Answering systems, as an extreme of the Information Retrieval field, could save lots of time and effort in satisfying a specific information need. In this regard, there are still many challenges to be resolved by current state-of-the-art systems as they cope with free texts. We propose a new hybrid question answering schema capable of answering questions with respect to different semantically related syntactically mismatched situations either in a structured or unstructured semantic format. We have exploited FrameNet and WordNet lexical resources and implemented the prototype system in a TREC-friendly fashion to obtain results comparable with outstanding participant systems in TREC 2004.

A FULLY AUTOMATED OFFLINE HANDWRITING RECOGNITION SYSTEM INCORPORATING RULE BASED NEURAL NETWORK VALIDATED SEGMENTATION AND HYBRID NEURAL NETWORK CLASSIFIER

In this paper we propose a fully automated offline handwriting recognition system that incorporates rule based segmentation, contour based feature extraction, neural network validation, a hybrid neural network classifier and a hamming neural network lexicon. The work is based on our earlier promising results in this area using heuristic segmentation and contour based feature extraction. The segmentation is done using many heuristic based set of rules in an iterative manner and finally followed by a neural network validation system. The extraction of feature is performed using both contour and structure based feature extraction algorithm. The classification is performed by a hybrid neural network that incorporates a hybrid combination of evolutionary algorithm and matrix based solution method. Finally a hamming neural network is used as a lexicon. A benchmark dataset from CEDAR has been used for training and testing.

MODULAR NEURAL NETWORK DESIGN FOR THE PROBLEM OF ALPHABETIC CHARACTER RECOGNITION

This paper reports on an experimental approach to nd a modularized articial neural network solution for the UCI letters recognition problem. Our experiments have been carried out in two parts. We investigate directed task decomposition using expert knowledge and clustering approaches to nd the subtasks for the modules of the network. We next investigate processes to combine the modules eectiv ely in a single decision process. After having found suitable modules through task decomposition we have found through further experimentation that when the modules are combined with decision tree supervision, their functional error is reduced signican tly to improve their combination through the decision process that has been implemented as a small multilayered perceptron. The experiments conclude with a modularized neural network design for this classication problem that has increased learning and generalization characteristics. The test results for this network are markedly better than a single or stand alone network that has a fully connected topology.

Finding optimal architecture and weights using evolutionary least square based learning

In this paper, we present a novel idea of implementing a growing neural network architecture using an evolutionary least square based algorithm. This paper focuses mainly on the following aspects, such as the heuristics of updating weights using an evolutionary least square based algorithm, finding the number of hidden neurons for a two layer feed forward multilayered perceptron (MLP), the stopping criteria for the algorithm and finally comparisons of the results with other traditional methods for searching optimal or near optimal solution in the multidimensional complex search space comprising the architecture and the weight variables. We applied our proposed algorithm for XOR data set, 10 bit odd parity problem and many real bench mark data set like handwriting dataset from CEDAR and breast cancer, heart disease data set from UCI ML repository. The comparison results, based on classification accuracy and the time complexity are discussed. We also discuss the issues of finding a probabilistic solution space as a starting point for the least square method and address the problems involving fitness breaking.

Least square method based evolutionary neural learning algorithm

We present a new idea of evolving weights for the artificial neural networks (ANNs), and propose a novel hybrid learning approach for the training of a feedforward ANN. The approach combines evolutionary algorithms with matrix solution methods such as Gram-Schmidt, etc., to achieve optimum weights for hidden and output layers. Our hybrid method is to apply the evolutionary algorithm in the first layer and the least square method in the second layer of the ANN. A two-layer network is considered. The hidden layer weights are evolved using the evolutionary algorithm. When a certain number of generation or error goal in terms of RMS/class error is reached, the training stops. We start with a small number of hidden neurons, and then the number is increased gradually. We applied our algorithm for XOR, 10-bit odd parity and handwritten segmented characters recognition problems. The implementation of the algorithm was done in MATLAB and C. Experiments show some promising results when compared with other evolutionary based algorithm only in terms of results in classification rate and time complexity.

Using association and overlapping time window approach to detect drug reaction signals

The problem with detecting adverse drug reactions (ADRs) from drugs is that they may not be obvious until long after they are widely prescribed. Part of the problem is these events are rare. This work describes an approach to signal detection of ADRs based on association rules (AR) in Australian drug safety data. This work was carried out using the Australian Adverse Drug Reactions Advisory Committee (ADRAC) database, which contains a hundred and thirty seven thousand records collected in 1972-2001 period. Many signal detection methods have been developed for drug safety data, most of which use a classical statistical approach. Some of these stratify the data using an ontology for reactions, but the application of drug ontologies to ADR signal detection methods has not been reported. We propose a novel approach for detecting various signal levels by using an overlapped windowing approach. The overlapping windows help to detect smooth transition of signal. We use association rules for measuring significant change over time for different hierarchical levels of drugs (using the anatomical-therapeutic-chemical (ATC) system of drug classification ontology) and their reactions based on the system organ classes (SOC) ontology. Using association rules and their strength for different levels in the drug and reaction hierarchy, helps in the detection of signals at particular levels in higher order using a bottom up approach. The results of a preliminary investigation of ADRAC data using our method demonstrate that this approach could produce a powerful and robust ADR signal detection method