Sanjeev S. Malalur

Multiple optimal learning factors for feed-forward networks

A batch training algorithm for feed-forward networks is proposed which uses Newtons method to estimate a vector of optimal learning factors, one for each hidden unit. Backpropagation, using this learning factor vector, is used to modify the hidden units input weights. Linear equations are then solved for the networks output weights. Elements of the new methods Gauss-Newton Hessian matrix are shown to be weighted sums of elements from the total networks Hessian. In several examples, the new method performs better than backpropagation and conjugate gradient, with similar numbers of required multiplies. The method performs as well as or better than Levenberg-Marquardt, with several orders of magnitude fewer multiplies due to the small size of its Hessian.

Feed-forward network training using optimal input gains

In this paper, an effective batch training algorithm is developed for feed-forward networks such as the multilayer perceptron. First, the effects of input transforms are reviewed and explained, using the concept of equivalent networks. Next, a non-singular diagonal transform matrix for the inputs is proposed. Use of this transform is equivalent to altering the input gains in the network. Newtons method is used to solve for the input gains and an optimal learning factor. In several examples, it is shown that the final algorithm is a reasonable compromise between first order training methods and Levenburg-Marquardt.

Multiple optimal learning factors for the multi-layer perceptron

Abstract A batch training algorithm is developed for a fully connected multi-layer perceptron, with a single hidden layer, which uses two-stages per iteration. In the first stage, Newton׳s method is used to find a vector of optimal learning factors (OLFs), one for each hidden unit, which is used to update the input weights. Linear equations are solved for output weights in the second stage. Elements of the new method׳s Hessian matrix are shown to be weighted sums of elements from the Hessian of the whole network. The effects of linearly dependent inputs and hidden units on training are analyzed and an improved version of the batch training algorithm is developed. In several examples, the improved method performs better than first order training methods like backpropagation and scaled conjugate gradient, with minimal computational overhead and performs almost as well as Levenberg–Marquardt, a second order training method, with several orders of magnitude fewer multiplications.

A Piecewise Linear Network Classifier

A piecewise linear network is discussed which classifies N-dimensional input vectors. The network uses a distance measure to assign incoming input vectors to an appropriate cluster. Each cluster has a linear classifier for generating class discriminants. A training algorithm is described for generating the clusters and discriminants. Theorems are given which relate the networks performance to that of nearest neighbor and k-nearest neighbor classifiers. It is shown that the error approaches Bayes error as the number of clusters and patterns per cluster approach infinity.

Analysis and improvement of multiple optimal learning factors for feed-forward networks

The effects of transforming the net function vector in the multilayer perceptron are analyzed. The use of optimal diagonal transformation matrices on the net function vector is proved to be equivalent to training the network using multiple optimal learning factors (MOLF). A method for linearly compressing large ill-conditioned MOLF Hessian matrices into smaller well-conditioned ones is developed. This compression approach is shown to be equivalent to using several hidden units per learning factor. The technique is extended to large networks. In simulations, the proposed algorithm performs almost as well as the Levenberg Marquardt algorithm with the computational complexity of a first order training algorithm.

A pseudospectral fusion approach to fingerprint matching

A prototype fingerprint verification system is described which combines the direction and density images into a complex pseudo-spectrum. Two methods for extracting the direction and density images are presented. The proposed feature extraction method is shown to be fast, efficient and robust Verification is achieved by correlation matching.

Properties of a Batch Training Algorithm for Feedforward Networks

We examine properties for a batch training algorithm known as the output weight optimization–hidden weight optimization (OWO–HWO). Using the concept of equivalent networks, we analyze the effect of input transformation on BP. We introduce new theory of affine invariance and partial affine invariance for neural networks and prove this property for OWO–HWO. Finally, we relate HWO to BP and show that every iteration of HWO is equivalent to BP applied to whitening transformed data. Experimental results validate the connection between OWO–HWO and OWO–BP.

Fingerprint Feature Compression Using Statistical Coding Techniques

We have proposed a new fingerprint feature compression scheme, which extracts the fingerprint feature using pseudo-spectral fusion approach and compresses the fingerprint feature using statistical coding technique. The compression scheme starts with fingerprint image acquisition, enhancement, and normalization. Further, the direction and density of the fingerprint image are extracted. Fusing these two direction and density data results finger print feature data. The fingerprint feature data is applied directly using a statistical coding technique to get compressed data. The Huffman coding and arithmetic coding is used for compression and found arithmetic coding performs better. This proposed compression scheme achieves nearly 85% compression rate, which is higher compared to other present techniques achieved 45% to 63% compression.