Pierre A. Devijver

Pattern recognition theory and applications

Statistical pattern recognition is now a mature discipline which has been successfully applied in several application domains. The primary goal in statistical pattern recognition is classification, where a pattern vector is assigned to one of a finite number of classes and each class is characterized by a probability density function on the measured features. A pattern vector is viewed as a point in the multidimensional space defined by the features. Design of a recognition system based on this paradigm requires careful attention to the following issues: type of classifier (single-stage vs. hierarchical), feature selection, estimation of classification error, parametric vs. nonparametric decision rules, and utilizing contextual information. Current research emphasis in pattern recognition is on designing efficient algorithms, studying small sample properties of various estimators and decision rules, implementing the algorithms on novel computer architecture, and incorporating context and domain-specific knowledge in decision making.

Baum's forward-backward algorithm revisited

In this note, we examine the forward-backward algorithm from the computational viewpoint of the underflow problem inherent in Baums (1972) original formulation. We demonstrate that the conversion of Baums computation of joint likelihoods into the computation of posterior probabilities results in essentially the same algorithm, except for the presence of a scaling factor suggested by Levinson et al. (1983) on rather heuristic grounds. The resulting algorithm is immune to the underflow problem, and Levinsons scaling method is given a theoretical justification. We also investigate the relationship between Baums algorithm and the recent algorithms of Askar and Derin (1981) and Devijver (1984).

Statistical Properties of Error Estimators in Performance Assessment of Recognition Systems

The problem of estimating the error probability of a given classification system is considered. Statistical properties of the empirical error count (C) and the average conditional error (R) estimators are studied. It is shown that in the large sample case the R estimator is unbiased and its variance is less than that of the C estimator. In contrast to conventional methods of Bayes error estimation the unbiasedness of the R estimator for a given classifier can be obtained only at the price of an additional set of classified samples. On small test sets the R estimator may be subject to a pessimistic bias caused by the averaging phenomenon characterizing the functioning of conditional error estimators.

PROBABILISTIC LABELING IN A HIDDEN SECOND ORDER MARKOV MESH

Abstract: In this paper, we address the pixel labeling problem under the assumption that contextual information in the picture is encoded in the transition probabilities of a second order Markov mesh random field. We show that the exponential complexity of the problem can be overcome by the virtue of a truly innocuous assumption that leads to a virtually optimal solution expressed in terms of a non-linear recurrence relation. We derive linear-time algorithms for computing the joint likelihood of the pixel label and past and present (and some look-ahead) image measurements. The smoothing and segmentation of coarsely quantized pictures are taken as example-problems to validate our assumption and demonstrate the effectiveness of the technique proposed.

Learning the parameters of a hidden Markov random field image model: A simple example

The paper outlines a unified treatment of the labeling and learning problems for the so-called hidden Markov chain model currently used in many speech recognition systems and the hidden Pickard random field image model (a small but interesting, causal sub-class of hidden Markov random field models). In both cases, labeling techniques are formulated in terms of Baum’s classical forward-backward recurrence formulae, and learning is accomplished by a specialization of the EM algorithm for mixture identification. Experimental results demonstrate that the approach is subjectively relevant to the image restoration and segmentation problems.

A note on ties in voting with the k-NN rule

Abstract It is shown that in the two-class case and when ties in voting are broken at random, the (2k′) and (2k′−1)-NN rules achieve identical large-sample performances. Slightly sharpened error-bounds are derived.

On the editing rate of the MULTIDIT algorithm

Abstract In a number of previous publications, we have stated, without proof, that the total fraction of samples discarded by the Multiedit algorithm is bounded from above by 2E1, where E1 is the 1-NNR error rate for the initial distributions. It is the purpose of this note to offer a more precise formulation together with a derivation of this assertion.

An application of the multiedit-condensing technique to the reference selection problem in a print recognition system

Abstract In this paper we address the elusive problem of selecting references (templates) for minimum distance classification when the number of pattern classes is very large. We argue that the multiedit/condensing technique offers an automatic solution to this problem which avoids the proliferation of references without impairing the recognition performance. The effectiveness of the approach is demonstrated by experimental results in a print recognition context. Suggestions are made about ways of circumventing problems of computational complexity.

An efficient estimator of pattern recognition system error probability

Abstract A new estimator of system error probability is proposed. The estimator combines the average conditional error method and the empirical error count method so that all the information available to the designer (test and reference data set samples and their labels) can be utilized most efficiently. It is shown that the proposed estimator is unbiased and has a lower variance than the average conditional error estimator proposed by Kittler and Devijver.(5)

A multiclass, k-NN approach to Bayes risk estimation

In this paper, the k-NN approach is used for the purpose of estimating the multiclass, 1-NN Bayes error bounds. We derive an estimator which is asymptotically unbiased, and whose variance can be controlled by the choice of k. The estimator appears to be very economic in its use of samples, and quite stable even in very small sample cases.