Shingo Tomita

An optimal orthonormal system for discriminant analysis

Abstract This paper proposes a new discriminant analysis with orthonormal coordinate axes of the feature space. In general, the number of coordinate axes of the feature space in the traditional discriminant analysis depends on the number of pattern classes. Therefore, the discriminatory capability of the feature space is limited considerably. The new discriminant analysis solves this problem completely. In addition, it is more powerful than the traditional one in so far as the discriminatory power and the mean error probability for coordinate axes are concerned. This is also shown by a numerical example.

A gabor filter-based method for recognizing handwritten numerals

Abstract We study a Gabor-filter-based method for handwritten numeral character recognition. The Gabor filter is based on a multi-channel filtering theory for processing visual information in the early stages of the human visual systems. The performance of the Gabor-filter-based method is demonstrated on the ETL-1 database. Experimental results show that the artificial neural-network classifier achieved the error rate of 2.34% for a test set of 7000 characters. Therefore, the Gabor-filter-based method should be considered in recognition of handwritten numeric characters.

A bootstrap technique for nearest neighbor classifier design

A bootstrap technique for nearest neighbor classifier design is proposed. Our primary interest in designing a classifier is in small training sample size situations. Conventional bootstrapping techniques sample the training samples with replacement. On the other hand, our technique generates bootstrap samples by locally combining original training samples. The nearest neighbor classifier is designed on the bootstrap samples and is tested on the test samples independent of training samples. The performance of the proposed classifier is demonstrated on three artificial data sets and one real data set. Experimental results show that the nearest neighbor classifier designed on the bootstrap samples outperforms the conventional k-NN classifiers as well as the edited 1-NN classifiers, particularly in high dimensions.

Recognition of handwritten numerals using Gabor features

We study a Gabor filter-based feature extraction method for handwritten numeral character recognition. The performance of the Gabor filter-based method is demonstrated on the ETL-1 database. Experimental results suggest that the Gabor filter-based method should be considered in recognition of handwritten numeric characters.

On the behavior of artificial neural network classifiers in high-dimensional spaces

It is widely believed in the pattern recognition field that when a fixed number of training samples is used to design a classifier, the generalization error of the classifier tends to increase as the number of features gets larger. In this paper, we discuss the generalization error of the artificial neural network (ANN) classifiers in high-dimensional spaces, under a practical condition that the ratio of the training sample size to the dimensionality is small. Experimental results show that the generalization error of ANN classifiers seems much less sensitive to the feature size than 1-NN, Parzen and quadratic classifiers.

Recognition of handprinted Chinese characters using Gabor features

A method for handprinted Chinese character recognition based on Gabor filters is proposed. The Gabor approach to character recognition is intuitively appealing because it is inspired by a multi-channel filtering theory for processing visual information in the early stages of the human visual system. The performance of a character recognition system using Gabor features is demonstrated on the ETL-8 character set. Mental results show that the Gabor features yielded an error rate of 2.4% versus the error rate of 4.4% obtained by using a popular feature extraction method.

A note on the orthonormal discriminant vector method for feature extraction

Abstract We propose a new feature extraction method based on the modified “plus e -take away f” algorithm and discuss an aspect of the optimization method used in sequential feature extraction, by comparing the proposed method with the orthonormal discriminant vector (ODV) method which belongs to a class of sequential feature extraction. It is shown from experimental results that the proposed method is superior to the ODV method in terms of the error probability.

On the estimation of a covariance matrix in designing Parzen classifiers

The design of the Parzen classifiers requires careful attention to the window-width as well as kernel covariance matrices. Although a considerable amount of effort has been devoted to the selection of the window-width, the problem of estimating kernel covariance matrices has received little attention in the past. In this paper we discuss the kernel covariance estimators for the design of the Parzen classifiers. We compare the performance of the Parzen classifiers based on several kernel covariance estimators such as the Toeplitz, Nesss and orthogonal expansion estimators on three artificial data sets. From experimental results, we recommend the use of the Toeplitz estimator, particularly in high-dimensional spaces.

Evaluation of the branch and bound algorithm for feature selection

Abstract Narendra and Fukunaga show that the branch and bound algorithm guarantees the optimal feature subset without evaluating all possible feature subsets, if the criterion function used satisfies the ‘monotonicity’ property. In this paper, we show that the algorithm works well in terms of the recognition rate, even if the property is not satisfied.

On a theoretical comparison between the orthonormal discriminant vector method and discriminant analysis

Abstract The performance of the orthonormal discriminant vector (ODV) method is discussed in comparison with discriminant analysis. The ODV method produces the features which maximize the Fisher criterion subject to the orthonormality of features. In contrast with discriminant analysis, the ODV method has no limitation on the maximum number of features to be extracted. From a theoretical viewpoint, it is proved that the ODV method is more powerful than discriminant analysis in terms of the Fisher criterion. The theoretical conclusion is experimentally verified using two real data sets.