Paulo Vinicius Wolski Radtke

Partially-supervised learning from facial trajectories for face recognition in video surveillance

Face recognition (FR) is employed in several video surveillance applications to determine if facial regions captured over a network of cameras correspond to a target individuals. To enroll target individuals, it is often costly or unfeasible to capture enough high quality reference facial samples a priori to design representative facial models. Furthermore, changes in capture conditions and physiology contribute to a growing divergence between these models and faces captured during operations. Adaptive biometrics seek to maintain a high level of performance by updating facial models over time using operational data. Adaptive multiple classifier systems (MCSs) have been successfully applied to video-to-video FR, where the face of each target individual is modeled using an ensemble of 2-class classifiers (trained using target vs. non-target samples). In this paper, a new adaptive MCS is proposed for partially-supervised learning of facial models over time based on facial trajectories. During operations, information from a face tracker and individual-specific ensembles is integrated for robust spatio-temporal recognition and for self-update of facial models. The tracker defines a facial trajectory for each individual that appears in a video, which leads to the recognition of a target individual if the positive predictions accumulated along a trajectory surpass a detection threshold for an ensemble. When the number of positive ensemble predictions surpasses a higher update threshold, then all target face samples from the trajectory are combined with non-target samples (selected from the cohort and universal models) to update the corresponding facial model. A learn-and-combine strategy is employed to avoid knowledge corruption during self-update of ensembles. In addition, a memory management strategy based on Kullback-Leibler divergence is proposed to rank and select the most relevant target and non-target reference samples to be stored in memory as the ensembles evolves. For proof-of-concept, a particular realization of the proposed system was validated with videos from Face in Action dataset. Initially, trajectories captured from enrollment videos are used for supervised learning of ensembles, and then videos from various operational sessions are presented to the system for FR and self-update with high-confidence trajectories. At a transaction level, the proposed approach outperforms baseline systems that do not adapt to new trajectories, and provides comparable performance to ideal systems that adapt to all relevant target trajectories, through supervised learning. Subject-level analysis reveals the existence of individuals for which self-updating ensembles with unlabeled facial trajectories provides a considerable benefit. Trajectory-level analysis indicates that the proposed system allows for robust spatio-temporal video-to-video FR, and may therefore enhance security and situation analysis in video surveillance.

Intelligent zoning design using multi-objective evolutionary algorithms

This paper discusses the use of multi objective evolutionaryalgorithms applied to the engineering of zoning forhandwriten recognition. Usually a task fulfilled by an humanexpert, zoning design relies on specific domain knowledgeand a trial and error process to select an adequatedesign. Our proposed approach to automatically define thezone design was tested and was able to define zoning strategiesthat performed better than our former strategy definedmanually.

Skew-sensitive boolean combination for adaptive ensembles - An application to face recognition in video surveillance

Abstract Several ensemble-based techniques have been proposed to design pattern recognition systems when data has imbalanced class distributions, although class proportions may change over time according to the operational environment. For instance, in video surveillance applications, face recognition (FR) is employed to detect the presence of target individuals of interest in potentially complex and changing environments. Systems for FR in video surveillance are typically designed a priori with a limited amount of reference target data and prior knowledge of underlying class distributions. However, the relatively proportion of target and non-target faces captured during operations varies over time. Estimating the actual proportion of data from the input data stream could allow to dynamically adapt ensembles to reflect operational conditions. In this paper, the selection and fusion of ensembles produced through Boolean Combination (BC) of classifiers is periodically adapted based on the class proportions estimated from input streams. BC techniques have been shown to efficiently integrate the responses of multiple diversified classifiers in the ROC space, yet the impact on performance of imbalanced data distributions is difficult to observe from ROC curves. Given a diversified pool of classifiers and a desired false positive rate ( fpr ), the new Skew-Sensitive Boolean Combination (SSBC) technique exploits the Precision-Recall Operating Characteristic (PROC) space, leading to a higher level of performance. A set of BCs of base classifiers is initially produced with imbalanced reference data in the PROC space, where each BC curve corresponds to different level of imbalance (a growing number of non-target samples versus a fixed number of target ones). Then, during operations, the closest adjacent levels of class imbalance are periodically estimated using the Hellinger distance between the data distribution of inputs and that of imbalance levels, and used to approximate the most accurate BC of classifiers from operational points of these curves. Simulation results on Faces In Action video surveillance data indicate that ensemble-based FR systems using the SSBC technique outperform the same systems using traditional BC techniques with Random Under-Sampling and One-Sided Selection. It allows to dynamically select BCs that provide a higher level of precision (and F1 value) for target individuals, and a significantly smaller difference between desired and actual fpr . Performance of this adaptive approach is also comparable to the costly full recalculation of BCs (as required by a BC technique to accommodate a specific level of imbalance), but for a computational complexity that is considerably lower. Finally, SSBC is shown to achieve a high level of discrimination between target and non-target individuals when face tracking is exploited to accumulate ensemble predictions for facial captures that correspond to a same person in the video scene.

A multi-objective memetic algorithm for intelligent feature extraction

This paper presents a methodology to generate representations for isolated handwritten symbols, modeled as a multi-objective optimization problem. We detail the methodology, coding domain knowledge into a genetic based representation. With the help of a model on the domain of handwritten digits, we verify the problematic issues and propose a hybrid optimization algorithm, adapted to needs of this problem. A set of tests validates the optimization algorithm and parameter settings in the models context. The results are encouraging, as the optimized solutions outperform the human expert approach on a known problem.

An Evaluation of Over-Fit Control Strategies for Multi-Objective Evolutionary Optimization

The optimization of classification systems is often confronted by the solution over-fit problem. Solution over-fit occurs when the optimized classifier memorizes the training data sets instead of producing a general model. This paper compares two validation strategies used to control the over-fit phenomenon in classifier optimization problems. Both strategies are implemented within the multi-objective NSGA-II and MOMA algorithms to optimize a projection distance classifier and a multiple layer perceptron neural network classifier, in both single and ensemble of classifier configurations. Results indicated that the use of a validation stage during the optimization process is superior to validation performed after the optimization process.

Incremental update of biometric models in face-based video surveillance

Video-based face recognition of individuals involves matching facial regions captured in video sequences against the model of individuals enrolled to a face recognition system. Due to a limited control over operational conditions, classification systems applied to face matching are confronted with complex pattern recognition environments that change over time. Therefore, the facial model of an individual tends to diverge from the underlying data distribution. Although a limited amount of reference data is often collected during initial enrollment, new samples often become available over time to update and refine models. In this paper, an adaptive ensemble of classifiers is proposed to update facial models in response to new reference samples. To avoid knowledge corruption linked to incremental learning of monolithic classifiers, and maintain a high level of performance, this ensemble exploits a learn-and-combine approach. In response to new reference samples, a new 2-class Probabilistic Fuzzy ARTMAP classifier is trained and combined to previously-trained classifiers in the ROC space. Iterative Boolean Combination is employed for fusion of 2-class classifiers of each individual in the decision space. Performance is assessed in terms of AUC accuracy and resource requirements under different incremental learning scenarios with new data extracted from the Faces in Action data set. Simulation results indicate that the proposed system significantly outperforms reference classifiers and ensembles for incremental learning.

Solution over-fit control in evolutionary multiobjective optimization of pattern classification systems

The optimization of many engineering systems is challenged by the solution over-fit to the data set used to evaluate potential solutions during the evolutionary process. The solution over-fit phenomenon is hard to detect and is especially prevalent in problems involving example-based training, such as pattern feature selection and pattern classi- fier design. For these applications, uncontrolled over-fit can lead to biased features being extracted and degraded classifier generalization abilities. This paper details the perfor- mance of a solution over-fit control strategy used in the multiobjective evolutionary optimization of a multileveled classification system. This control, embedded within a solution validation procedure, minimizes the over-fit effects without modifying the dom- inance relation used in the processing of candidate solutions. Extensive experimental analysis using multiobjective genetic and memetic algorithms demonstrates both the need and the efficiency of the proposed over-fit control for pattern classification systems optimization.

Intelligent feature extraction for ensemble of classifiers

This paper presents a two-level approach to create ensemble of classifiers based on intelligent feature extraction and multi-objective genetic optimization. The first stage optimizes a set of representations, which is used to create classifiers. The second stage then optimizes the ensembles aggregated classifiers. To assess the approachs feasibility, a set of tests with isolated handwritten digits is performed. The experimental results encourage further researches in this direction, as the optimized ensemble of classifiers outperforms the single classifier approach.

Adaptive Ensemble Selection for Face Re-identification under Class Imbalance

Systems for face re-identification over a network of video surveillance cameras are designed with a limited amount of reference data, and may operate under complex environments. Furthermore, target individuals provide a small proportion of the facial captures for design and during operations, and these proportions may change over time according to operational conditions. Given a diversified pool of base classifiers and a desired false positive rate (fpr), the Skew-Sensitive Boolean Combination (SSBC) technique allows to adapt the selection of ensembles based on changes to levels of class imbalance, as estimated from the input video stream. Initially, a set of BCs for the base classifiers is produced in the ROC space, where each BC curve corresponds to reference data with a different level of imbalance. Then, during operations, class imbalance is periodically estimated using the Hellinger distance between the data distribution of inputs and that of imbalance levels, and used to approximate the most accurate BC of classifiers among operational points of these curves viewed in the precision-recall space. Simulation results on real-world video surveillance data indicate that, compared to traditional approaches, FR systems based on SSBC allow to select BCs that provide a higher level of precision for target individuals, and a significantly smaller difference between desired and actual fpr. Performance of this adaptive approach is also comparable to full recalculation of BCs (for a specific level of imbalance), but for a considerably lower complexity. Using face tracking, a high level of discrimination between target and non-target individuals may be achieved by accumulating SSBC predictions for faces captured corresponding to a same track in video footage.

Using the RRT algorithm to optimize classification systems for handwritten digits and letters

Multi-objective genetic algorithms have been often used to optimize classification systems, but little is discussed on their computational cost to solve such problems. This paper optimizes a classification system with an annealing based approach, the Record-to-Record Travel algorithm. Results obtained are compared to those obtained with a multi-objective genetic algorithm in the same approach. Experiments are performed with isolated handwritten digits and uppercase letters, demonstrating both the effectiveness and lower computational cost of the annealing based approach.