Pavel Pudil

Multistage pattern recognition with reject option

The idea of constructing a multistage pattern classification system with reject option is presented and conditions in terms of upper bounds of the cost of higher-stage measurements for a multistage classifier to give lower decision risk than a single-stage classifier are derived.<<ETX>>

Efficient Feature Subset Selection and Subset Size Optimization

A broad class of decision-making problems can be solved by learning approach. This can be a feasible alternative when neither an analytical solution exists nor the mathematical model can be constructed. In these cases the required knowledge can be gained from the past data which form the so-called learning or training set. Then the formal apparatus of statistical pattern recognition can be used to learn the decision-making. The first and essential step of statistical pattern recognition is to solve the problem of feature selection (FS) ormore generally dimensionality reduction (DR). The problem of feature selection in statistical pattern recognition will be of primary focus in this chapter. The problem fits in the wider context of dimensionality reduction (Section 2) which can be accomplished either by a linear or nonlinear mapping from the measurement space to a lower dimensional feature space, or by measurement subset selection. This chapter will focus on the latter (Section 3). The main aspects of the problem as well as the choice of the right feature selection tools will be discussed (Sections 3.1 to 3.3). Several optimization techniques will be reviewed, with emphasis put to the framework of sequential selection methods (Section 4). Related topics of recent interest will be also addressed, including the problem of subset size determination (Section 4.7), search acceleration through hybrid algorithms (Section 5), and the problem of feature selection stability and feature over-selection (Section 6).

Mixtures of Product Components Versus Mixtures of Dependence Trees

Mixtures of product components assume independence of variables given the index of the component. They can be efficiently estimated from data by means of EM algorithm and have some other useful properties. On the other hand, by considering mixtures of dependence trees, we can explicitly describe the statistical relationship between pairs of variables at the level of individual components and therefore approximation power of the resulting mixture may essentially increase. However, we have found in application to classification of numerals that both models perform comparably and the contribution of dependence-tree structures to the log-likelihood criterion decreases in the course of EM iterations. Thus the optimal estimate of dependence-tree mixture tends to reduce to a simple product mixture model.

Evaluation of the effectiveness of features selected by discriminant analysis methods

Abstract New theoretical and practical results concerning the use of discriminant analysis for feature selection are presented in the paper. Numerical values for the eigenvalues of the matrix SW W −1 S B (within-class and between-class scatter matrices) are investigated. An analytic expression for their minimum value representing the minimum effectiveness is derived. Differences between real values and these minimum values are important for the evaluation of the effectiveness of features and thus for feature selection.

Identifying Corporate Performance Factors Based on FeatureSelection in Statistical Pattern Recognition: METHODS,APPLICATION, INTERPRETATION

This publication summarizes and extends methodology of feature selection (FS) and pattern recognition in search for competitiveness factors and methodology of corporate financial performance (CFP) measurement. Several methods were evaluated and Dependency-Aware Feature Ranking combined with non-linear regression model were applied. Also, this publication suggests and verifies methodology of interpretation results of the FS methods. For start was employed multidimensional linear regression, succeeded by clustering companies according to the factors identified by FS into homogenous groups, dividing them into quartiles based on their CFP and identifying similar values of the factors. This way was captured the non-linearity in the data.