Matti Aksela

Using diversity of errors for selecting members of a committee classifier

Diversity of classifiers is generally accepted as being necessary for combining them in a committee. Quantifying diversity of classifiers, however, is difficult as there is no formal definition thereof. Numerous measures have been proposed in literature, but their performance is often know to be suboptimal. Here several common methods are compared with a novel approach focusing on the diversity of the errors made by the member classifiers. Experiments with combining classifiers for handwritten character recognition are presented. The results show that the approach of diversity of errors is beneficial, and that the novel exponential error count measure is capable of consistently finding an effective member classifier set.

Comparison of classifier selection methods for improving committee performance

Combining classifiers is an effective way of improving classification performance. In many situations it is possible to construct several classifiers with different characteristics. Selecting the member classifiers with the best individual performance can be shown to be suboptimal in several cases, and hence there exists a need to attempt to find effective member classifier selection methods. In this paper six selection criteria are discussed and evaluated in the setting of combining classifiers for isolated handwritten character recognition. A criterion focused on penalizing many classifiers making the same error, the exponential error count, is found to be able to produce the best selections.

Adaptive combination of adaptive classifiers for handwritten character recognition

In this paper we examine the feasibility of combining two distinct layers of on-line adaptation for improving overall handwritten character recognition performance. These two approaches are adaptive classifiers and an adaptive committee used to combine them. On-line adaptive handwritten character classifiers are first discussed and the significant performance enhancements they can provide illustrated. We then examine the benefits from combining classifiers for this task, adaptive and non-adaptive, and present an adaptive committee structure suitable for this doubly adaptive framework. Experiments in combining the two adaptation approaches to form an adaptive committee consisting of adaptive member classifiers are described. The results show that while adaptation of the individual classifiers provides on average the most benefit in comparison to the non-adaptive reference level, the use of an adaptive combination of adaptive classifiers is still capable of enhancing the recognition performance by a significant margin. The usefulness of the proposed doubly adaptive approach is in this paper demonstrated in the domain of on-line handwritten character recognition, but we argue that the proposed methodology could also be applied to other application domains.

Dynamically expanding context as committee adaptation method in on-line recognition of handwritten Latin characters

We have developed an adaptive handwriting recognizer for isolated Latin characters in which the adaptive behavior is based on the dynamically expanding context (DEC) algorithm. In our current system, the outputs of a set of static classifiers are combined in a committee machine, whose rules are adapted. Every misclassified character gives rise to adding a new DEC rule to the rule set of the committee. When the existing rules fail to produce a correct recognition output, more and more context information is utilized in forming the new DEC rules. Not only the first-ranking outputs from the member classifiers but also the second-ranking ones can be taken into account when forming the DEC rules. In the experiments described in this paper, various options in the implementation of the DEC committee classifier are evaluated. The results of the experiments show that the system is capable of fast adaptation to the users handwriting and lead to lowered recognition error rates.

Methods for adaptive combination of classifiers with application to recognition of handwritten characters

Abstract.This paper discusses two techniques for improving the recognition accuracy for online handwritten character recognition: committee classification and adaptation to the user. Combining classifiers is a common method for improving recognition performance. Improvements are possible because the member classifiers may make different errors. Much variation exists in handwritten characters, and adaptation is one feasible way of dealing with such variation. Even though adaptation is usually performed for single classifiers, it is also possible to use adaptive committees. Some novel adaptive committee structures, namely, the dynamically expanding context (DEC), modified current best learning (MCBL), and class-confidence critic combination (CCCC), are presented and evaluated. They are shown to be able to improve on their member classifiers, with CCCC offering the best performance. Also, the effect of having either more or less diverse sets of member classifiers is considered.

Adaptive local subspace classifier in on-line recognition of handwritten characters

Subsystems for online recognition of handwriting are needed in personal digital assistants (PDA) and other portable handheld devices. We have developed a recognition system which enhances its accuracy by applying continuous adaptation to the users writing style. The forms of adaptation we have experimented with take place simultaneously with the normal operation of the system and therefore, there is no need for separate training period of the device. The present implementation uses dynamic time warping (DTW) in matching the input characters with the stored prototypes. The DTW algorithm implemented with dynamic programming (DP) is, however both time and memory consuming. In our current research we have experimented with methods that transform the elastic templates to pixel images which can then be recognized by using statistical or neural classification. The particular neural classifier we have used is the local subspace classifier (LSC) of which we have developed an adaptive version.

Application of Adaptive Committee Classifiers in On-Line Character Recognition

There are two main approaches to classifier adaptation. A single adaptive classfier can be used, or an adaptive committee of classifiers whose members can be either adaptive or non-adaptive. We have experimented with some approaches to adaptive committee operations, including the Dynamically Expanding Context (DEC) and the Modified Current-Best-Learning (MCBL) approaches. In the experiments of this paper the feasibility of using an adaptive committee classifier is explored and tested with on-line character recognition. The results clearly show that the use of adaptive committeees can improve on the recognition results, both in comparison to the individual member classifiers and the non-adaptive reference committee.

Class-confidence critic combining

This paper discusses a combination of two techniques for improving the recognition accuracy of on-line handwritten character recognition: committee classification and adaptation to the user. A novel adaptive committee structure, namely the class-confidence critic combination (CCCC) scheme, is presented and evaluated. It is shown to be able to improve significantly on its member classifiers. Also the effect of having either more or less diverse sets of member classifiers is considered.