Nicoletta Noceti

Using Object Affordances to Improve Object Recognition

The problem of object recognition has not yet been solved in its general form. The most successful approach to it so far relies on object models obtained by training a statistical method on visual features obtained from camera images. The images must necessarily come from huge visual datasets, in order to circumvent all problems related to changing illumination, point of view, etc. We hereby propose to also consider, in an object model, a simple model of how a human being would grasp that object (its affordance). This knowledge is represented as a function mapping visual features of an object to the kinematic features of a hand while grasping it. The function is practically enforced via regression on a human grasping database. After describing the database (which is publicly available) and the proposed method, we experimentally evaluate it, showing that a standard object classifier working on both sets of features (visual and motor) has a significantly better recognition rate than that of a visual-only classifier.

Ask the Image: Supervised Pooling to Preserve Feature Locality

In this paper we propose a weighted supervised pooling method for visual recognition systems. We combine a standard Spatial Pyramid Representation which is commonly adopted to encode spatial information, with an appropriate Feature Space Representation favoring semantic information in an appropriate feature space. For the latter, we propose a weighted pooling strategy exploiting data supervision to weigh each local descriptor coherently with its likelihood to belong to a given object class. The two representations are then combined adaptively with Multiple Kernel Learning. Experiments on common benchmarks (Caltech-256 and PASCAL VOC-2007) show that our image representation improves the current visual recognition pipeline and it is competitive with similar state-of-art pooling methods. We also evaluate our method on a real Human-Robot Interaction setting, where the pure Spatial Pyramid Representation does not provide sufficient discriminative power, obtaining a remarkable improvement.

Learning common behaviors from large sets of unlabeled temporal series

This paper is about extracting knowledge from large sets of videos, with a particular reference to the video-surveillance application domain. We consider an unsupervised framework and address the specific problem of modeling common behaviors from long-term collection of instantaneous observations. Specifically, such data describe dynamic events and may be represented as time series in an appropriate space of features. Starting off from a set of data meaningful of the common events in a given scenario, the pipeline we propose includes a data abstraction level, that allows us to process different data in a homogeneous way, and a behavior modeling level, based on spectral clustering. At the end of the pipeline we obtain a model of the behaviors which are more frequent in the observed scene, represented by a prototypical behavior, which we call a cluster candidate. We report a detailed experimental evaluation referring to both benchmark datasets and on a complex set of data collected in-house. The experiments show that our method compares very favorably with other approaches from the recent literature. In particular the results we obtain prove that our method is able to capture meaningful information and discard noisy one from very heterogeneous datasets with different levels of prior information available.

Combined Motion and Appearance Models for Robust Object Tracking in Real-Time

This paper proposes a tracking architecture that finds a trade-off between accuracy and efficiency, via a combined solution of motion and appearance information. We explore the use of color features into a tracking pipeline based on Kalman filtering. The devised architecture is made of simple modules, combined to reach a robust final result, while keeping the computation cost low (we perform

Structured multi-class feature selection with an application to face recognition

20

Online Space-Variant Background Modeling With Sparse Coding

fps). The method has been evaluated on three benchmark datasets and is currently under use on real video-surveillance systems, reporting very good tracking results.

Biological movement detector enhances the attentive skills of humanoid robot iCub

In this paper we address the problem of structured feature selection in a multi-class classification setting. Our goal is to select groups of features meaningful to all classes simultaneously, and to this purpose we derive a new formulation of Group LASSO - the MC-GrpLASSO - and a solution of the obtained functional based on proximal methods. We then apply the algorithm to a typical multi-class problem - face recognition. On this respect we focus on finding an effective and fast to compute (that is, sparse) representation of faces, detected in low quality videos of unconstrained environments. We start from a classical over-complete representation based on Local Binary Patterns (LBPs), descriptors endowed with a characteristic internal structure that can be preserved by selecting features in groups. We present an extensive experimental analysis on two benchmark datasets, MOBO and Choke Point, and on a more complex set of data acquired in-house over a large temporal span. We compare our results with state-of-the-art approaches and show the superiority of our method in terms of both performances and sparseness of the obtained solution.

Cognition Helps Vision: Recognizing Biological Motion Using Invariant Dynamic Cues

In this paper, we propose a sparse coding approach to background modeling. The obtained model is based on dictionaries which we learn and keep up to date as new data are provided by a video camera. We observe that, without dynamic events, video frames may be seen as noisy data belonging to the background. Over time, such background is subject to local and global changes due to variable illumination conditions, camera jitter, stable scene changes, and intermittent motion of background objects. To capture the locality of some changes, we propose a space-variant analysis where we learn a dictionary of atoms for each image patch, the size of which depends on the background variability. At run time, each patch is represented by a linear combination of the atoms learnt online. A change is detected when the atoms are not sufficient to provide an appropriate representation, and stable changes over time trigger an update of the current dictionary. Even if the overall procedure is carried out at a coarse level, a pixel-wise segmentation can be obtained by comparing the atoms with the patch corresponding to the dynamic event. Experiments on benchmarks indicate that the proposed method achieves very good performances on a variety of scenarios. An assessment on long video streams confirms our method incorporates periodical changes, as the ones caused by variations in natural illumination. The model, fully data driven, is suitable as a main component of a change detection system.

Semi-supervised learning of sparse representations to recognize people spatial orientation

Detecting human activity in the scene is a fundamental skill for robotics. Most current methods of human detection from video analysis rely on appearance or shape features, thus exhibiting severe limitations as clutter or scene complexity grow, for instance when humans are using tools. We propose a way to overcome these limitations by exploiting a motionbased human detection system relying on the regularities of human kinematics. Our method, which is implemented as an open-source software module and integrated in the humanoid robot iCub software framework, extracts relevant features of biological motion in a computationally efficient way and feeds them to the attentional system of the robot. As a result the robot can rapidly direct its attention toward the human agents in the scene, even when they are hidden or partially covered by the tools they are using. The paper describes in detail the software framework supporting the autonomous learning of a discrimination policy between biological and non-biological motion observed in a scene. Then, it provides a quantitative validation of the classification performances on a batch dataset acquired by the iCub cameras and in a scenario where both training and classification are performed online. Last, it presents experiments on the integration of the module with the iCub attention system, demonstrating the ability of the robot to selectively and rapidly redeploy its fixation point on the human activity in the scene. The experimental results show that the proposed system can reliably enable the robot to focus its attention on human activity, a fundamental first step to allow for a deeper understanding of the observed action and a careful planning of an interaction strategy with a human partner.

Humans in groups: The importance of contextual information for understanding collective activities

This paper considers the problem of designing computational models of the primitives that are at the basis of the visual perception of motion in humans. The main contribution of this work is to establish a connection between cognitive science observations and empirical computational modeling. We take inspiration from the very first stage of the human development, and address the problem of understanding the presence of biological motion in the scene. To this end, we investigate the use of coarse motion descriptors composed by low-level features inspired by the Two-Thirds Power Law. In the experimental analysis, we first discuss the validity of the Two-Thirds Power Law in the context of video analysis, where, to the best of our knowledge, it has not found application so far. Second, we show a preliminary investigation on the use of a very simple motion model for characterizing biological motion with respect to non-biological dynamic events.