Matteo Taiana

Tracking objects with generic calibrated sensors: An algorithm based on color and 3D shape features

We present a color and shape based 3D tracking system suited to a large class of vision sensors. The method is applicable, in principle, to any known calibrated projection model. The tracking architecture is based on particle filtering methods where each particle represents the 3D state of the object, rather than its state in the image, therefore overcoming the nonlinearity caused by the projection model. This allows the use of realistic 3D motion models and easy incorporation of self-motion measurements. All nonlinearities are concentrated in the observation model so that each particle projects a few tens of special points onto the image, on (and around) the 3D objects surface. The likelihood of each state is then evaluated by comparing the color distributions inside and outside the objects occluding contour. Since only pixel access operations are required, the method does not require the use of image processing routines like edge/feature extraction, color segmentation or 3D reconstruction, which can be sensitive to motion blur and optical distortions typical in applications of omnidirectional sensors to robotics. We show tracking applications considering different objects (balls, boxes), several projection models (catadioptric, dioptric, perspective) and several challenging scenarios (clutter, occlusion, illumination changes, motion and optical blur). We compare our methodology against a state-of-the-art alternative, both in realistic tracking sequences and with ground truth generated data.

The HDA+ Data Set for Research on Fully Automated Re-identification Systems

There are no available datasets to evaluate integrated Pedestrian Detectors and Re-Identification systems, and the standard evaluation metric for Re-Identification (Cumulative Matching Characteristic curves) does not properly assess the errors that arise from integrating Pedestrian Detectors with Re-Identification (False Positives and Missed Detections). Real world Re-Identification systems require Pedestrian Detectors to be able to function automatically and the integration of Pedestrian Detector algorithms with Re-Identification produces errors that must be dealt with. We provide not only a dataset that allows for the evaluation of integrated Pedestrian Detector and Re-Identification systems but also sample Pedestrian Detection data and meaningful evaluation metrics and software, such as to make it “one-click easy” to test your own Re-Identification algorithm in an Integrated PD+REID system without having to implement a Pedestrian Detector algorithm yourself. We also provide body-part detection data on top of the manually labeled data and the Pedestrian Detection data, such as to make it trivial to extract your features from relevant local regions (actual body-parts). Finally we provide camera synchronization data to allow for the testing of inter-camera tracking algorithms. We expect this dataset and software to be widely used and boost research in integrated Pedestrian Detector and Re-Identification systems, bringing them closer to reality.

A multi-camera video dataset for research on high-definition surveillance

We present a fully labelled image sequence dataset for benchmarking video surveillance algorithms. The dataset was acquired from 13 indoor cameras distributed over three floors of one building, recording simultaneously for 30 minutes. The dataset was specially designed and labelled to tackle the person detection and re-identification problems. Around 80 persons participated in the data collection, most of them appearing in more than one camera. The dataset is heterogeneous: there are three distinct types of cameras (standard, high and very high resolution), different view types (corridors, doors, open spaces) and different frame rates. This diversity is essential for a proper assessment of the robustness of video analytics algorithms in different imaging conditions. We illustrate the application of pedestrian detection and re-identification algorithms to the given dataset, pointing out important criteria for benchmarking and the impact of high-resolution imagery on the performance of the algorithms.

An Improved Labelling for the INRIA Person Data Set for Pedestrian Detection

Data sets are a fundamental tool for comparing detection algorithms, fostering advances in the state of the art. The INRIA person data set is very popular in the Pedestrian Detection community, both for training detectors and reporting results. Yet, the labelling of its test set has some limitations: some of the pedestrians are not labelled, there is no specific label for the ambiguous cases and the information on the visibility ratio of each person is missing. We present a new labelling that overcomes such limitations and show that it can be used to evaluate the performance of detection algorithms in a more truthful way.

Predictive tracking across occlusions in the iCub robot

In humans the tracking of a visual moving target across occlusions is not made with continuous smooth pursuit. The tracking stops when the object is occluded and one or two saccades are made to the other side of the occluder to anticipate when and where the object reappears. This paper describes a methodology for the implementation of such a behavior in a robotic platform - the iCub. We use the RLS algorithm for the on-line estimation and prediction of the target trajectory and a vision based object tracker capable of detecting the occlusion and the reappearance of an object. This system demonstrates predictive ability for tracking across an occlusion with a biologically-plausible behavior.

Sample-Based 3D Tracking of Colored Objects: A Flexible Architecture

This paper presents a method for 3D model-based tracking of colored objects using a sampling methodology. The problem is formulated in a Monte Carlo filtering approach, whereby the state of an object is re presented by a set of hypotheses. The main originality of this work is an observation model consisting in the comparison of the color information in some sampling points around the target’s hypothetical edges. On the contrary to existing approaches the method does not need to explicitly compute edges in the video stream, thus dealing well with optical or motion blur. The method does not require the projection of the full 3D object on the image, but just of some selected points around the target’s boundaries. This a llows a flexible and modular architecture illustrated by experiments performed with different objects (balls and boxes), camera models (perspective, catadioptric, dioptric) and tracking methodologies (particle and Kalman filtering) .

3D Tracking by Catadioptric Vision Based on Particle Filters

This paper presents a robust tracking system for autonomous robots equipped with omnidirectional cameras. The proposed method uses a 3D shape and color-based object model. This allows to tackle difficulties that arise when the tracked object is placed above the ground plane floor. Tracking under these conditions has two major difficulties: first, observation with omnidirectional sensors largely deforms the targets shape; second, the object of interest embedded in a dynamic scenario may suffer from occlusion, overlap and ambiguities. To surmount these difficulties, we use a 3D particle filterto represent the targets state space: position and velocity with respect to the robot. To compute the likelihood of each particle the following features are taken into account: i) image color; ii) mismatch between targets color and background color. We test the accuracy of the algorithm in a RoboCup Middle Size League scenario, both with static and moving targets.

On the purity of training and testing data for learning: The case of pedestrian detection

Abstract The training and the evaluation of learning algorithms depend critically on the quality of data samples. We denote as pure the samples that identify clearly and without any ambiguity the class of objects of interest. For instance, in pedestrian detection algorithms, we consider as pure samples the ones containing persons who are fully visible and are imaged at a good resolution (larger than the detector window in size). The exclusive use of pure samples entails two kinds of problems. In training, it biases the detector to neglect slightly occluded and small sized samples (which we denote as impure), thus reducing its detection rate in a real world application. In testing, it leads to the unfair evaluation and comparison of different detectors since slightly impure samples, when detected, can be accounted for as false positives. In this paper we study how a sensible use of impure samples can benefit both the training and the evaluation of pedestrian detection algorithms. We improve the labelling of one of the most widely used pedestrian data sets (INRIA) taking into account the degree of sample impurity. We observe that including partially occluded pedestrians in the training improves performance, not only on partially visible examples, but also on the fully visible ones. Furthermore, we found that including pedestrians imaged at low resolutions is beneficial for detecting pedestrians in the same range of heights, leaving the performance on pure samples unchanged. However, including samples with too high a grade of impurity degrades the performance, thus a careful balance must be found. The proposed labelling will allow further studies on the role of impure samples in training pedestrian detectors and on devising fairer comparison metrics between different algorithms.

A Window-Based Classifier for Automatic Video-Based Reidentification

The vast quantity of visual data generated by the rapid expansion of large scale distributed multicamera networks, makes automated person detection and reidentification (RE-ID) essential components of modern surveillance systems. However, the integration of automated person detection and RE-ID algorithms is not without problems, and the errors arising in this integration must be measured (e.g., detection failures that may hamper the RE-ID performance). In this paper, we present a window-based classifier based on a recently proposed architecture for the integration of pedestrian detectors and RE-ID algorithms, that takes the output of any bounding-box RE-ID classifier and exploits the temporal continuity of persons in video streams. We evaluate our contributions on a recently proposed dataset featuring 13 high-definition cameras and over 80 people, acquired during 30 min at rush hour in an office space scenario. We expect our contributions to drive research in integrated pedestrian detection and RE-ID systems, bringing them closer to practical applications.