Federico Pedersini

Multi-camera systems

In this article, we present some simple and effective techniques for accurately calibrating a multi-camera acquisition system. The proposed methods were proven to be capable of accurate results even when using very simple calibration target sets and low-cost imaging devices, such as standard TV-resolution cameras connected to commercial frame-grabbers. In fact, the performance of our calibration approach yielded results that were about the same as that of other traditional calibration methods based on large 3D target sets. The proposed calibration strategy is based on a multi-view multi-camera approach. This was based on the analysis of a number of views of a simple calibration target-set placed in different (unknown) positions. Furthermore, the method is based on a self-calibration approach, which can refine the a priori knowledge of the world coordinates of the targets (even when such information is very poor) while estimating the parameters of the camera model. Finally we proposed a method, to make the calibration technique adaptive through the analysis of natural scene features, allowing the camera parameters to hold accurate throughout the acquisition session in the presence of parameter drift.

Estimation and compensation of subpixel edge localization error

We propose and analyze a method for improving the performance of subpixel edge localization (EL) techniques through compensation of the systematic portion of the localization error. The method is based on the estimation of the EL characteristic through statistical analysis of a test image and is independent of the EL technique in use.

3D area matching with arbitrary multiview geometry

In this article we present a general and robust approach to the problem of close-range 3D reconstruction of objects from stereo correspondence of luminance patches. The method is largely independent on the camera geometry, and can employ an arbitrary number of CCD cameras. The robustness of the approach is due to the physicality of the matching process, which is performed in the 3D space. In fact, both 3D location and local orientation of the surface patches are estimated, so that the geometric distortion can be accounted for. The method takes into account the viewer-dependent radiometric distortion as well. The method has been implemented with a calibrated set of three standard TV-resolution CCD cameras. Experiments on a variety of real scenes have been conducted with satisfactory results. Quantitative and qualitative results are reported.

Object-based sound synthesis for virtual environments-using musical acoustics

We show how to construct an object-based environment for sound generation, whose objects can be individually synthesized and which can interact with each other through modelling of a potential interaction topology. We show how this interaction topology can be mode dynamic and time-varying. We also discuss how we envision an object-based environment that integrates geometric, radiometric, and intrinsic/extrinsic acoustic properties. We finally illustrate our first results towards modelling of complex sound generation systems.

Automatic monitoring and 3D reconstruction applied to cultural heritage

Abstract In this article we present our global approach to the problem of accurate 3D measurement and reconstruction of 3D works of art using a calibrated multi-camera system. In particular, we illustrate a simple and effective adaptive technique for the self-calibration of CCD-based multi-camera acquisition systems with minimum a-prior information. We also propose a general and robust approach to the problem of close-range partial 3D reconstruction of objects from stereo-correspondences. Finally, we introduce a method for performing an accurate patchworking of the partial reconstructions, based on 3D curve matching.

Egomotion estimation of a multicamera system through line correspondences

In this paper we propose a method for estimating the egomotion of a calibrated multi-camera system from an analysis of the luminance edges. The method works entirely in the 3D space as all edges of each one set of views are previously localized, matched and back-projected onto the object space. In fact, it searches for the rigid motion that best merges the sets of 3D contours extracted from each one of the multi-views. The method uses both straight and curved 3D contours.

3D motion estimation of a trinocular system for a full-3D object reconstruction

Motion estimation of a calibrated multi-ocular acquisition system can be performed either on two-dimensional data, by applying a rigidity constraint to a set of matched points on monocular views, or on three-dimensional data, by determining the rigid motion that best overlaps homologous 3D points obtained through stereo matching and back-projection. We propose, analyze and compare these two possible solutions, and present a low-cost high-accuracy full-3D reconstruction system based on multiple trinocular views at standard TV resolution.

Accurate feature detection and matching for the tracking of calibration parameters in multi-camera acquisition systems

The 3D reconstructions quality of multiple-camera acquisition systems is strongly influenced by the accuracy of the camera calibration procedure. The acquisition of long sequences is, in fact, very sensitive to mechanical shocks, vibrations and thermal changes on cameras and supports, as they could result in a significant drift of the camera parameters. In this paper we propose a technique which is able to keep track of the camera parameters and, whenever possible, to correct them accordingly. This technique does not need any a-priori knowledge or test objects to be placed in the scene, but exploits features that are already present in the scene itself. In fact it performs an accurate detection, matching and back-projection of luminance corners and spots in the scene space. Experimental results on real sequences are reported in order to prove the ability of the proposed technique to detect a change in the calibration and to re-calibrate the camera setup with an accuracy that depends on the number of available feature points.

Combined motion and edge analysis for a layer-based representation of image sequences

We propose a method for the motion-congruent segmentation of image sequences based on both motion field and luminance information. In order to do so, the affine motion models are determined by analyzing the motion field through a clustering procedure, while their regions of validity are determined by an MRF-based region estimator. This last block performs a pixel-wise re-assignment of a limited number of affine models (those determined via clustering).

The ORIGAMI project: advanced tools and techniques for high-end mixing and interaction between real and virtual content

ORIGAMI is an EU-funded IST project with the goal to develop advanced tools and new production techniques for high-quality mixing of real and virtual content for film and TV productions. In particular, the project proposes new techniques for 3D camera calibration and tracking and for 3D scene reconstruction and plenoptic modelling. Such solutions will support a variety of different object classes and application scenarios, including single-camera modelling of static environments, and multi-camera modelling of dynamic studio scenes. The developed methods will be integrated into a new software framework, which is being developed within the project. ORIGAMI also proposes novel techniques that enable real and virtual actors to interact with each other and with the real and virtual elements of the extended set.