Federico Pernici

Metric 3D reconstruction and texture acquisition of surfaces of revolution from a single uncalibrated view

Image analysis and computer vision can be effectively employed to recover the three-dimensional structure of imaged objects, together with their surface properties. In this paper, we address the problem of metric reconstruction and texture acquisition from a single uncalibrated view of a surface of revolution (SOR). Geometric constraints induced in the image by the symmetry properties of the SOR structure are exploited to perform self-calibration of a natural camera, 3D metric reconstruction, and texture acquisition. By exploiting the analogy with the geometry of single axis motion, we demonstrate that the imaged apparent contour and the visible segments of two imaged cross sections in a single SOR view provide enough information for these tasks. Original contributions of the paper are: single view self-calibration and reconstruction based on planar rectification, previously developed for planar surfaces, has been extended to deal also with the SOR class of curved surfaces; self-calibration is obtained by estimating both camera focal length (one parameter) and principal point (two parameters) from three independent linear constraints for the SOR fixed entities; the invariant-based description of the SOR scaling function has been extended from affine to perspective projection. The solution proposed exploits both the geometric and topological properties of the transformation that relates the apparent contour to the SOR scaling function. Therefore, with this method, a metric localization of the SOR occluded parts can be made, so as to cope with them correctly. For the reconstruction of textured SORs, texture acquisition is performed without requiring the estimation of external camera calibration parameters, but only using internal camera parameters obtained from self-calibration.

Object Tracking by Oversampling Local Features.

In this paper, we present the ALIEN tracking method that exploits oversampling of local invariant representations to build a robust object/context discriminative classifier. To this end, we use multiple instances of scale invariant local features weakly aligned along the object template. This allows taking into account the 3D shape deviations from planarity and their interactions with shadows, occlusions, and sensor quantization for which no invariant representations can be defined. A non-parametric learning algorithm based on the transitive matching property discriminates the object from the context and prevents improper object template updating during occlusion. We show that our learning rule has asymptotic stability under mild conditions and confirms the drift-free capability of the method in long-term tracking. A real-time implementation of the ALIEN tracker has been evaluated in comparison with the state-of-the-art tracking systems on an extensive set of publicly available video sequences that represent most of the critical conditions occurring in real tracking environments. We have reported superior or equal performance in most of the cases and verified tracking with no drift in very long video sequences.

Exploiting distinctive visual landmark maps in pan-tilt-zoom camera networks

Pan-tilt-zoom (PTZ) camera networks have an important role in surveillance systems. They have the ability to direct the attention to interesting events that occur in the scene. One method to achieve such behavior is to use a process known as sensor slaving: one (or more) master camera monitors a wide area and tracks moving targets so as to provide the positional information to one (or more) slave camera. The slave camera can thus point towards the targets in high resolution. In this paper we describe a novel framework exploiting a PTZ camera network to achieve high accuracy in the task of relating the feet position of a person in the image of the master camera, to his head position in the image of the slave camera. Each camera in the network can act as a master or slave camera, thus allowing the coverage of wide and geometrically complex areas with a relatively small number of sensors. The proposed framework does not require any 3D known location to be specified, and allows to take into account both zooming and target uncertainties. Quantitative results show good performance in target head localization, independently from the zooming factor in the slave camera. An example of cooperative tracking approach exploiting with the proposed framework is also presented.

Towards on-line saccade planning for high-resolution image sensing

This paper considers the problem of designing an active observer to plan a sequence of decisions regarding what target to look at, through a foveal-sensing action. We propose a framework in which a pan/tilt/zoom (PTZ) camera schedules saccades in order to acquire high resolution images (at least one) of as many moving targets as possible before they leave the scene. An intelligent choice of the order of sensing the targets can significantly reduce the total dead-time wasted by the active camera and, consequently, its cycle time. The grabbed images provide meaningful identification imagery of distant targets which are not recognizable in a wide angle view. We cast the whole problem as a particular kind of dynamic discrete optimization. In particular, we will show that the problem can be solved by modelling the attentional gaze control as a novel on-line dynamic vehicle routing problem (DVRP) with deadlines. Moreover we also show how multi-view geometry can be used for evaluating the cost of high resolution image sensing with a PTZ camera.Congestion analysis experiments are reported proving the effectiveness of the solution in acquiring high resolution images of a large number of moving targets in a wide area. The evaluation was conducted with a simulation using a dual camera system in a master-slave configuration. Camera performances are also empirically tested in order to validate how the manufacturers specification deviates from our model using an off-the-shelf PTZ camera.

Acquisition of high-resolution images through on-line saccade sequence planning

This paper considers the problem of scheduling an active observer to visit as many targets in an area of surveillance as possible. We show how it is possible to plan a sequence of decisions regarding what target to look at through such a foveal-sensing action. We propose a framework in which a pan/tilt/zoom camera executes saccades in order to visit, and acquire high resolution images (at least one) of, as many moving targets as possible before they leave the scene. An intelligent choice of the order of sensing the targets can significantly reduce the total dead-time wasted by the active camera and, consequently, its cycle time. We cast the whole problem into a dynamic discrete optimization framework. In particular, we will show that the problem can be solved by modeling the attentional gaze control as a kinetic traveling salesperson problem whose solution is approximated by iteratively solving time dependent orienteering problems.Congestion analysis experiments are reported demonstrating the effectiveness of the solution in acquiring high resolution images of a large number of moving targets in a wide area. The evaluation was conducted with a simulation of a dual camera system in a master-slave configuration. We also report on preliminary experiments conducted using live cameras in a real surveillance environment.

Accurate self-calibration of two cameras by observations of a moving person on a ground plane

A calibration algorithm of two cameras using observations of a moving person is presented. Similar methods have been proposed for self-calibration with a single camera, but internal parameter estimation is only limited to the focal length. Recently it has been demonstrated that principal point supposed in the center of the image causes inaccuracy of all estimated parameters. Our method exploits two cameras, using image points of head and foot locations of a moving person, to determine for both cameras the focal length and the principal point. Moreover with the increasing number of cameras there is a demand of procedures to determine their relative placements. In this paper we also describe a method to find the relative position and orientation of two cameras: the rotation matrix and the translation vector which describe the rigid motion between the coordinate frames fixed in two cameras. Results in synthetic and real scenes are presented to evaluate the performance of the proposed method.

Uncalibrated 3D metric reconstruction and flattened texture acquisition from a single view of a surface of revolution

We describe a geometric approach for reconstructing 3D textured graphical models of surface of revolution (SOR) objects from a single uncalibrated view Our approach is based on the fact that, for the object class of interest, the structure of the scene provides enough constraints for camera calibration even from a single view. Reconstruction (up to a scaling factor) of 3D shape is complemented with the extraction of flattened 2D texture, so as to support visual retrieval from 2D/3D cues anti to generate realistic 3D visualization models. The approach developed is quite simple, yet accurate and robust; its applications range from the preservation, analysis and classification of cultural heritage, to advanced graphics and multimedia.

Accurate Automatic Localization of Surfaces of Revolution for Self-Calibration and Metric Reconstruction

In this paper, we address the problem of the automatic metric reconstruction Surface of Revolution (SOR) from a single uncalibrated view. The apparent contour and the visible portions of the imaged SOR cross sections are extracted and classified. The harmonic homology that models the image projection of the SOR is also estimated. The special care devoted to accuracy and robustness with respect to outliers makes the approach suitable for automatic camera calibration and metric reconstruction from single uncalibrated views of a SOR. Robustness and accuracy are obtained by embedding a graph-based grouping strategy (Euclidean Minimum Spanning Tree) into an Iterative Closest Point framework for projective curve alignment at multiple scales. Classification of SOR curves is achieved through a 2-dof voting scheme based on a pencil of conics novel parametrization. The main contribution of this work is to extend the domain of automatic single view reconstruction from piecewise planar scenes to scenes including curved surfaces, thus allowing to create automatically realistic image models of man-made objects. Experimental results with real images taken from the internet are reported, and the effectiveness and limitations of the approach are discussed.

Distant targets identification as an on-line dynamic vehicle routing problem using an active-zooming camera

This paper considers the problem of modeling an active observer to plan a sequence of decisions regarding what target to look at, through a foveal-sensing action. The gathered images by the active observer provides meaningful identification imagery of distant targets which are not recognizable in a wide angle view. We propose a framework in which a pan/tilt/zoom (PTZ) camera schedules saccades in order to acquire high resolution images of as many moving targets as possible before they leave the scene. We cast the whole problem as a particular kind of dynamic discrete optimization, specially as a novel on-line dynamic vehicle routing problem (DVRP) with deadlines. We show that using an optimal choice for the sensing order of targets the total time spent in visiting the targets by the active camera can be significantly reduced. To show the effectiveness of our approach we apply congestion analysis to a dual camera system in a master-slave configuration. We report that our framework gives good results in monitoring wide areas with little extra costs with respect to approaches using a large number of cameras.

FaceHugger: the ALIEN tracker applied to faces

This paper proposes an online tracking method which has been inspired by studying the effects of Scale Invariant Feature Transform (SIFT) when applied to objects assumed to be flat even though they are not. The consequent deviation from flatness induces nuisance factors that act on the feature representation in a manner for which no general local invariants can be computed, such as in the case of occlusion, sensor quantization and casting shadows. However, if features are over-represented, they can provide the necessary information to build online, a robust object/context discriminative classifier. This is achieved based on weakly aligned multiple instance local features in a sense that will be made clear in the rest of this paper. According to this observation, we present a non parametric online tracking by detection approach that yields state of the art performance. Specific tests on video sequences of faces show excellent long-term tracking performance in unconstrained videos.