Giorgio Fumera

A theoretical and experimental analysis of linear combiners for multiple classifier systems

In this paper, a theoretical and experimental analysis of linear combiners for multiple classifier systems is presented. Although linear combiners are the most frequently used combining rules, many important issues related to their operation for pattern classification tasks lack a theoretical basis. After a critical review of the framework developed in works by Turner and Ghosh [1996], [1999] on which our analysis is based, we focus on the simplest and most widely used implementation of linear combiners, which consists of assigning a nonnegative weight to each individual classifier. Moreover, we consider the ideal performance of this combining rule, i.e., that achievable when the optimal values of the weights are used. We do not consider the problem of weights estimation, which has been addressed in the literature. Our theoretical analysis shows how the performance of linear combiners, in terms of misclassification probability, depends on the performance of individual classifiers, and on the correlation between their outputs. In particular, we evaluate the ideal performance improvement that can be achieved using the weighted average over the simple average combining rule and investigate in what way it depends on the individual classifiers. Experimental results on real data sets show that the behavior of linear combiners agrees with the predictions of our analytical model. Finally, we discuss the contribution to the state of the art and the practical relevance of our theoretical and experimental analysis of linear combiners for multiple classifier systems.

Reject option with multiple thresholds

can be obtained using the so-called “reject” option. Namely, the patterns that are the most likely to be misclassified are rejected (i.e., they are not classified); they are then handled by more sophisticated procedures (e.g., a manual classification is performed). However, handling high reject rates is usually too time-consuming for application purposes. Therefore, a trade-off between error and reject is mandatory. The formulation of the best error-reject trade-off and the related optimal reject rule was given by Chow [1]. According to Chow’s rule, a pattern x is rejected if: max P P T k N k i = ( ) = ( ) < 1, , | | K ω ω x x (3)

Security Evaluation of PatternClassifiers under Attack

Pattern classification systems are commonly used in adversarial applications, like biometric authentication, network intrusion detection, and spam filtering, in which data can be purposely manipulated by humans to undermine their operation. As this adversarial scenario is not taken into account by classical design methods, pattern classification systems may exhibit vulnerabilities, whose exploitation may severely affect their performance, and consequently limit their practical utility. Extending pattern classification theory and design methods to adversarial settings is thus a novel and very relevant research direction, which has not yet been pursued in a systematic way. In this paper, we address one of the main open issues: evaluating at design phase the security of pattern classifiers, namely, the performance degradation under potential attacks they may incur during operation. We propose a framework for empirical evaluation of classifier security that formalizes and generalizes the main ideas proposed in the literature, and give examples of its use in three real applications. Reported results show that security evaluation can provide a more complete understanding of the classifiers behavior in adversarial environments, and lead to better design choices.

Multiple classifier systems for robust classifier design in adversarial environments

Pattern recognition systems are increasingly being used in adversarial environments like network intrusion detection, spam filtering and biometric authentication and verification systems, in which an adversary may adaptively manipulate data to make a classifier ineffective. Current theory and design methods of pattern recognition systems do not take into account the adversarial nature of such kind of applications. Their extension to adversarial settings is thus mandatory, to safeguard the security and reliability of pattern recognition systems in adversarial environments. In this paper we focus on a strategy recently proposed in the literature to improve the robustness of linear classifiers to adversarial data manipulation, and experimentally investigate whether it can be implemented using two well known techniques for the construction of multiple classifier systems, namely, bagging and the random subspace method. Our results provide some hints on the potential usefulness of classifier ensembles in adversarial classification tasks, which is different from the motivations suggested so far in the literature.

Security evaluation of biometric authentication systems under real spoofing attacks

Multimodal biometric systems are commonly believed to be more robust to spoofing attacks than unimodal systems, as they combine information coming from different biometric traits. Recent work has shown that multimodal systems can be misled by an impostor even by spoofing only one biometric trait. This result was obtained under a `worst-case` scenario, by assuming that the distribution of fake scores is identical to that of genuine scores (i.e. the attacker is assumed to be able to perfectly replicate a genuine biometric trait). This assumption also allows one to evaluate the robustness of score fusion rules against spoofing attacks, and to design robust fusion rules, without the need of actually fabricating spoofing attacks. However, whether and to what extent the `worst-case` scenario is representative of real spoofing attacks is still an open issue. In this study, we address this issue by an experimental investigation carried out on several data sets including real spoofing attacks, related to a multimodal verification system based on face and fingerprint biometrics. On the one hand, our results confirm that multimodal systems are vulnerable to attacks against a single biometric trait. On the other hand, they show that the `worst-case` scenario can be too pessimistic. This can lead to two conservative choices, if the `worst-case` assumption is used for designing a robust multimodal system. Therefore developing methods for evaluating the robustness of multimodal systems against spoofing attacks, and for designing robust ones, remain a very relevant open issue.

Support Vector Machines with Embedded Reject Option

In this paper, the problem of implementing the reject option in support vector machines (SVMs) is addressed. We started by observing that methods proposed so far simply apply a reject threshold to the outputs of a trained SVM. We then showed that, under the framework of the structural risk minimisation principle, the rejection region must be determined during the training phase of a classifier. By applying this concept, and by following Vapniks approach, we developed a maximum margin classifier with reject option. This led us to a SVM whose rejection region is determined during the training phase, that is, a SVM with embedded reject option. To implement such a SVM, we devised a novel formulation of the SVM training problem and developed a specific algorithm to solve it. Preliminary results on a character recognition problem show the advantages of the proposed SVM in terms of the achievable error-reject trade-off.

A Theoretical Analysis of Bagging as a Linear Combination of Classifiers

We apply an analytical framework for the analysis of linearly combined classifiers to ensembles generated by bagging. This provides an analytical model of bagging misclassification probability as a function of the ensemble size, which is a novel result in the literature. Experimental results on real data sets confirm the theoretical predictions. This allows us to derive a novel and theoretically grounded guideline for choosing bagging ensemble size. Furthermore, our results are consistent with explanations of bagging in terms of classifier instability and variance reduction, support the optimality of the simple average over the weighted average combining rule for ensembles generated by bagging, and apply to other randomization-based methods for constructing classifier ensembles. Although our results do not allow to compare bagging misclassification probability with the one of an individual classifier trained on the original training set, we discuss how the considered theoretical framework could be exploited to this aim.

An Experimental Comparison of Classifier Fusion Rules for Multimodal Personal Identity Verification Systems

In this paper, an experimental comparison between fixed and trained fusion rules for multimodal personal identity verification is reported. We focused on the behaviour of the considered fusion methods for ensembles of classifiers exhibiting significantly different performance, as this is one of the main characteristics of multimodal biometrics systems. The experiments were carried out on the XM2VTS database, using eight experts based on speech and face data. As fixed fusion methods, we considered the sum, majority voting, and order statistics based rules. The considered trained methods are the Behaviour Knowledge Space and the weighted averaging of classifiers outputs.

Support vector machines for remote sensing image classification

In the last decade, the application of statistical and neural network classifiers to remote-sensing images has been deeply investigated. Therefore, performances, characteristics, and pros and cons of such classifiers are quite well known, even from remote-sensing practitioners. In this paper, we present the application to remote-sensing image classification of a new pattern recognition technique recently introduced within the framework of the Statistical Learning Theory developed by V. Vapnik and his co-workers, namely, the Support Vector Machines (SVMs). In section 1, the main theoretical foundations of SVMs are presented. In section 2, experiments carried out on a data set of multisensor remote-sensing images are described, with particular emphasis on the design and training phase of a SVM. In section 3, the experimental results are reported, together with a comparison between the performances of SVMs, neural network, and k-NN classifiers.

A multiple component matching framework for person re-identification

Person re-identification consists in recognizing an individual that has already been observed over a network of cameras. It is a novel and challenging research topic in computer vision, for which no reference framework exists yet. Despite this, previous works share similar representations of human body based on part decomposition and the implicit concept of multiple instances. Building on these similarities, we propose a Multiple Component Matching (MCM) framework for the person reidentification problem, which is inspired by Multiple Component Learning, a framework recently proposed for object detection [3]. We show that previous techniques for person re-identification can be considered particular implementations of our MCM framework. We then present a novel person re-identification technique as a direct, simple implementation of our framework, focused in particular on robustness to varying lighting conditions, and show that it can attain state of the art performances.