Isabelle Guyon

A training algorithm for optimal margin classifiers

A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented. The technique is applicable to a wide variety of the classification functions, including Perceptrons, polynomials, and Radial Basis Functions. The effective number of parameters is adjusted automatically to match the complexity of the problem. The solution is expressed as a linear combination of supporting patterns. These are the subset of training patterns that are closest to the decision boundary. Bounds on the generalization performance based on the leave-one-out method and the VC-dimension are given. Experimental results on optical character recognition problems demonstrate the good generalization obtained when compared with other learning algorithms.

Comparison of classifier methods: a case study in handwritten digit recognition

This paper compares the performance of several classifier algorithms on a standard database of handwritten digits. We consider not only raw accuracy, but also training time, recognition time, and memory requirements. When available, we report measurements of the fraction of patterns that must be rejected so that the remaining patterns have misclassification rates less than a given threshold.

Signature verification using a siamese' time delay neural network

This paper describes an algorithm for verification of signatures written on a pen-input tablet. The algorithm is based on a novel, artificial neural network, called a Siamese neural network. This network consists of two identical sub-networks joined at their outputs. During training the two sub-networks extract features from two signatures, while the joining neuron measures the distance between the two feature vectors. Verification consists of comparing an extracted feature vector with a stored feature vector for the signer. Signatures closer to this stored representation than a chosen threshold are accepted, all other signatures are rejected as forgeries.

UNIPEN project of on-line data exchange and recognizer benchmarks

We report the status of the UNIPEN project of data exchange and recognizer benchmarks started two years ago at the initiative of the International Association of Pattern Recognition (Technical Committee 11). The purpose of the project is to propose and implement solutions to the growing need of handwriting samples for online handwriting recognizers used by pen-based computers. Researchers from several companies and universities have agreed on a data format, a platform of data exchange and a protocol for recognizer benchmarks. The online handwriting data of concern may include handprint and cursive from various alphabets (including Latin and Chinese), signatures and pen gestures. These data will be compiled and distributed by the Linguistic Data Consortium. The benchmarks will be arbitrated the US National Institute of Standards and Technologies. We give a brief introduction to the UNIPEN format. We explain the protocol of data exchange and benchmarks.

Design of a neural network character recognizer for a touch terminal

Abstract We describe a system which can recognize digits and uppercase letters handprinted on a touch terminal. A character is input as a sequence of [ x(t), y(t) ] coordinates, subjected to very simple preprocessing, and then classified by a trainable neural network. The classifier is analogous to “time delay neural networks” previously applied to speech recognition. The network was trained on a set of 12,000 digits and uppercase letters, from approximately 250 different writers, and tested on 2500 such characters from other writers. Classification accuracy exceeded 96% on the test examples.

APPLICATIONS OF NEURAL NETWORKS TO CHARACTER RECOGNITION

Among the many applications that have been proposed for neural networks, character recognition has been one of the most successful. Compared to other methods used in pattern recognition, the advantage of neural networks is that they offer a lot of flexibility to the designer, i.e. expert knowledge can be introduced into the architecture to reduce the number of parameters determined by training by examples. In this paper, a general introduction to neural network architectures and learning algorithms commonly used for pattern recognition problems is given. The design of a neural network character recognizer for on-line recognition of handwritten characters is then described in detail.

Design of a linguistic postprocessor using variable memory length Markov models

We describe a linguistic postprocessor for character recognizers. The central module of our system is a trainable variable memory length Markov model (VLMM) that predicts the next character given a variable length window of past characters. The overall system is composed of several finite state automata, including the main VLMM and a proper noun VLMM. The best model reported in the literature (Brown et al., 1992) achieves 1.75 bits per character on the Brown corpus. On that same corpus, our model, trained on 10 times less data, reaches 2.19 bits per character and is 200 times smaller (/spl sime/160,000 parameters). The model was designed for handwriting recognition applications but could also be used for other OCR problems and speech recognition.

Writer-adaptation for on-line handwritten character recognition

The authors have designed a writer-adaptable character recognition system for online characters entered on a touch terminal. It is based on a Time Delay Neural Network (TDNN) that is pre-trained on examples from many writers to recognize digits and uppercase letters. The TDNN without its last layer serves as a preprocessor for an optimal hyperplane classifier that can be easily retrained to peculiar writing styles. This combination allows for fast writer-dependent learning of new letters and symbols. The system is memory and speed efficient.<<ETX>>

Computer aided cleaning of large databases for character recognition

A method for computer-aided cleaning of undesirable patterns in large training databases has been developed. The method uses the trainable classifier itself, to point out patterns that are suspicious, and should be checked by the human supervisor. While suspicious patterns that are meaningless or mislabeled are considered garbage, and removed from the database, the remaining patterns, like ambiguous or atypical, represent valid patterns that are hard to learn and should be kept in the database. By using the method of pattern cleaning, combined with an emphasizing scheme applied on the patterns that are hard to learn, the error rate on the test set has been reduced by half, in the case of the database of handwritten lowercase characters entered on a touch terminal. The classifier is based on a time delay neural network (TDNN).<<ETX>>

Recognition-based segmentation of on-line run-on handprinted words: Input vs. output segmentation

Abstract The performance of two methods for recognition-based segmentation of strings of on-line handprinted capital Latin characters is reported. The input strings consist of a time-ordered sequence of X, Y coordinates, punctuated by pen-lifts. The methods are designed to work in “run-on mode” where there is no constraint on the spacing between characters. While both methods use a neural network recognition engine and a graph-algorithmic post-processor, their approaches to segmentation are quite different. The first method, which we call INSEG (for input segmentation), uses a combination of heuristics to identify particular pen-lifts as tentative segmentation points. The second method, which we call OUTSEG (for output segmentation), relies on the empirically trained recognition engine for both recognizing characters and identifying relevant segmentation points. The best results are obtained with the INSEG method: 11% error on handprinted words from an 80,000 word dictionary.