Roberto Toldo

Robust Multiple Structures Estimation with J-Linkage

This paper tackles the problem of fitting multiple instances of a model to data corrupted by noise and outliers. The proposed solution is based on random sampling and conceptual data representation. Each point is represented with the characteristic function of the set of random models that fit the point. A tailored agglomerative clustering, called J-linkage, is used to group points belonging to the same model. The method does not require prior specification of the number of models, nor it necessitate parameters tuning. Experimental results demonstrate the superior performances of the algorithm.

A Bag of Words Approach for 3D Object Categorization

In this paper we propose a novel framework for 3D object categorization. The object is modeled it in terms of its sub-parts as an histogram of 3D visual word occurrences. We introduce an effective method for hierarchical 3D object segmentation driven by the minima rule that combines spectral clustering --- for the selection of seed-regions --- with region growing based on fast marching. The front propagation is driven by local geometry features, namely the Shape Index. Finally, after the coding of each object according to the Bag-of-Words paradigm, a Support Vector Machine is learnt to classify different objects categories. Several examples on two different datasets are shown which evidence the effectiveness of the proposed framework.

The bag of words approach for retrieval and categorization of 3D objects

In this paper, we propose a novel framework for 3D object retrieval and categorization. The object is modeled in terms of its subparts as an histogram of 3D visual word occurrences. We introduce an effective method for hierarchical 3D object segmentation driven by the minima rule that combines spectral clustering—for the selection of seed-regions—with region growing based on fast marching. Descriptors attached to the regions allow the definition of the visual words. After coding of each object according to the Bag-of-Words paradigm, retrieval can be performed by matching with a suitable kernel, or categorization by learning a Support Vector Machine. Several examples on the Aim@Shape watertight dataset and on the Tosca dataset demonstrate the versatility of the proposed method in working with either 3D objects with articulated shape changes or partially occluded or compound objects. Results are encouraging as shown by the comparison with other methods for each of the analyzed scenarios.

Hierarchical structure-and-motion recovery from uncalibrated images

We describe a hierarchical structure-from-motion pipeline.No information is needed beside images themselves.The pipeline proved successful in real-world tasks. This paper addresses the structure-and-motion problem, that requires to find camera motion and 3D structure from point matches. A new pipeline, dubbed Samantha, is presented, that departs from the prevailing sequential paradigm and embraces instead a hierarchical approach. This method has several advantages, like a provably lower computational complexity, which is necessary to achieve true scalability, and better error containment, leading to more stability and less drift. Moreover, a practical autocalibration procedure allows to process images without ancillary information. Experiments with real data assess the accuracy and the computational efficiency of the method.

Photo-consistent planar patches from unstructured cloud of points

Planar patches are a very compact and stable intermediate representation of 3D scenes, as they are a good starting point for a complete automatic reconstruction of surfaces. This paper presents a novel method for extracting planar patches from an unstructured cloud of points that is produced by a typical structure and motion pipeline. The method integrates several constraints inside J-linkage, a robust algorithm for multiple models fitting. It makes use of information coming both from the 3D structure and the images. Several results show the effectiveness of the proposed approach.

Editor's Choice Article: Image-consistent patches from unstructured points with J-linkage

Going from unstructured cloud of points to surfaces is a challenging problem. However, as points are produced by a structure-and-motion pipeline, image-consistency is a powerful clue that comes to the rescue. In this paper we present a method for extracting planar patches from an unstructured cloud of points, based on the detection of image-consistent planar patches with J-linkage, a robust algorithm for multiple model fitting. The method integrates several constraints inside J-linkage, optimizes the position of the points with regard to image-consistency and deploys a hierarchical processing scheme that decreases the computational load. With respect to previous work this approach has the advantage of starting from sparse data. Several results show the effectiveness of the proposed approach.

Automatic Estimation of the Inlier Threshold in Robust Multiple Structures Fitting

This paper tackles the problem of estimating the inlier threshold in RANSAC-like approaches to multiple models fitting. An iterative approach finds the maximum of a score function which resembles the Silhouette index used in clustering validation. Although several methods have been proposed to solve this problem for the single model case, this is the first attempt to address multiple models. Experimental results demonstrate the performances of the algorithm.

Samantha: Towards Automatic Image-Based Model Acquisition

In this paper we describe SAMANTHA, a Structure and Motion pipeline from images which is both more robust and computationally cheaper than current competing approaches. Pictures are organized into a hierarchical tree which has single images as leaves and partial reconstructions as internal nodes. The method proceeds bottom up until it reaches the root node, corresponding to the final result. This framework is one order of magnitude faster than sequential approaches, inherently parallel, less sensitive to the error accumulation causing drift and truly uncalibrated, not needing EXIF metadata to be present in pictures. We have verified the quality of our reconstructions both qualitatively producing compelling point clouds and quantitatively, comparing them with laser scans serving as ground truth. We also show how to automatically extract a meaningful collection of planar patches obtaining a compact, stable representation of scenes.

A Matrix Decomposition Perspective on Calibrated Photometric Stereo

Leveraging on recent advances in robust matrix decomposition, we revisit Lambertian photometric stereo as a robust low-rank matrix recovery problem with both missing and corrupted entries, tailoring Grasta and R-GoDec to normal surface estimation. A method to automatically detect shadows is proposed. The performance of different robust matrix completion techniques are analyzed on the challenging DiLiGenT datasets.

Geometric registration of remotely sensed data with SAMIR

The commercial market offers several software packages for the registration of remotely sensed data through standard one-to-one image matching. Although very rapid and simple, this strategy does not take into consideration all the interconnections among the images of a multi-temporal data set. This paper presents a new scientific software, called Satellite Automatic Multi-Image Registration (SAMIR), able to extend the traditional registration approach towards multi-image global processing. Tests carried out with high-resolution optical (IKONOS) and high-resolution radar (COSMO-SkyMed) data showed that SAMIR can improve the registration phase with a more rigorous and robust workflow without initial approximations, user’s interaction or limitation in spatial/spectral data size. The validation highlighted a sub-pixel accuracy in image co-registration for the considered imaging technologies, including optical and radar imagery.