Umberto Castellani

Sparse points matching by combining 3D mesh saliency with statistical descriptors

This paper proposes new methodology for the detection and matching of salient points over several views of an object. The process is composed by three main phases. In the first step, detection is carried out by adopting a new perceptually‐inspired 3D saliency measure. Such measure allows the detection of few sparse salient points that characterize distinctive portions of the surface. In the second step, a statistical learning approach is considered to describe salient points across different views. Each salient point is modelled by a Hidden Markov Model (HMM), which is trained in an unsupervised way by using contextual 3D neighborhood information, thus providing a robust and invariant point signature. Finally, in the third step, matching among points of different views is performed by evaluating a pairwise similarity measure among HMMs. An extensive and comparative experimental session has been carried out, considering real objects acquired by a 3D scanner from different points of view, where objects come from standard 3D databases. Results are promising, as the detection of salient points is reliable, and the matching is robust and accurate.

Coarse-to-fine low-rank structure-from-motion

We address the problem of deformable shape and motion recovery from point correspondences in multiple perspective images. We use the low-rank shape model, i.e. the 3D shape is represented as a linear combination of unknown shape bases. We propose a new way of looking at the low-rank shape model. Instead of considering it as a whole, we assume a coarse-to-fine ordering of the deformation modes, which can be seen as a model prior. This has several advantages. First, the high level of ambiguity of the original low-rank shape model is drastically reduced since the shape bases can not anymore be arbitrarily re-combined. Second, this allows us to propose a coarse-to-fine reconstruction algorithm which starts by computing the mean shape and iteratively adds deformation modes. It directly gives the sought after metric model, thereby avoiding the difficult upgrading step required by most of the other methods. Third, this makes it possible to automatically select the number of deformation modes as the reconstruction algorithm proceeds. We propose to incorporate two other priors, accounting for temporal and spatial smoothness, which are shown to improve the quality of the recovered model parameters. The proposed model and reconstruction algorithm are successfully demonstrated on several videos and are shown to outperform the previously proposed algorithms.

Using hidden Markov models and wavelets for face recognition

In this paper, a new system for face recognition is proposed, based on hidden Markov models (HMM) and wavelet coding. A sequence of overlapping sub-images is extracted from each face image, computing the wavelet coefficients for each of them. The whole sequence is then modelled by using hidden Markov models. The proposed method is compared with a DCT coefficient-based approach (Kohir et al. (1998)), showing comparable results. By using an accurate model selection procedure, we show that results proposed in Kohir can be improved even more. The obtained results outperform all results presented in the literature on the Olivetti Research Laboratory (ORL) face database, reaching a 100% recognition rate. This performance proves the suitability of HMM to deal with the new JPEG2000 image compression standard.

Learning class-specific descriptors for deformable shapes using localized spectral convolutional networks

In this paper, we propose a generalization of convolutional neural networks (CNN) to non‐Euclidean domains for the analysis of deformable shapes. Our construction is based on localized frequency analysis (a generalization of the windowed Fourier transform to manifolds) that is used to extract the local behavior of some dense intrinsic descriptor, roughly acting as an analogy to patches in images. The resulting local frequency representations are then passed through a bank of filters whose coefficient are determined by a learning procedure minimizing a task‐specific cost. Our approach generalizes several previous methods such as HKS, WKS, spectral CNN, and GPS embeddings. Experimental results show that the proposed approach allows learning class‐specific shape descriptors significantly outperforming recent state‐of‐the‐art methods on standard benchmarks.

Supervised learning of bag-of-features shape descriptors using sparse coding

We present a method for supervised learning of shape descriptors for shape retrieval applications. Many content‐based shape retrieval approaches follow the bag‐of‐features (BoF) paradigm commonly used in text and image retrieval by first computing local shape descriptors, and then representing them in a ‘geometric dictionary’ using vector quantization. A major drawback of such approaches is that the dictionary is constructed in an unsupervised manner using clustering, unaware of the last stage of the process (pooling of the local descriptors into a BoF, and comparison of the latter using some metric). In this paper, we replace the clustering with dictionary learning, where every atom acts as a feature, followed by sparse coding and pooling to get the final BoF descriptor. Both the dictionary and the sparse codes can be learned in the supervised regime via bi‐level optimization using a task‐specific objective that promotes invariance desired in the specific application. We show significant performance improvement on several standard shape retrieval benchmarks.

A Bag of Words Approach for 3D Object Categorization

In this paper we propose a novel framework for 3D object categorization. The object is modeled it in terms of its sub-parts as an histogram of 3D visual word occurrences. We introduce an effective method for hierarchical 3D object segmentation driven by the minima rule that combines spectral clustering --- for the selection of seed-regions --- with region growing based on fast marching. The front propagation is driven by local geometry features, namely the Shape Index. Finally, after the coding of each object according to the Bag-of-Words paradigm, a Support Vector Machine is learnt to classify different objects categories. Several examples on two different datasets are shown which evidence the effectiveness of the proposed framework.

Geo-located image analysis using latent representations

Image categorization is undoubtedly one of the most challenging open problems faced in computer vision, far from being solved by employing pure visual cues. Recently, additional textual ldquotagsrdquo can be associated to images, enriching their semantic interpretation beyond the pure visual aspect, and helping to bridge the so-called semantic gap. One of the latest class of tags consists in geo-location data, containing information about the geographical site where an image has been captured. Such data motivate, if not require, novel strategies to categorize images, and pose new problems to focus on. In this paper, we present a statistical method for geo-located image categorization, in which categories are formed by clustering geographically proximal images with similar visual appearance. The proposed strategy permits also to deal with the geo-recognition problem, i.e., to infer the geographical area depicted by images with no available location information. The method lies in the wide literature on statistical latent representations, in particular, the probabilistic latent semantic analysis (pLSA) paradigm has been extended, introducing a latent aspect which characterizes peculiar visual features of different geographical zones. Experiments on categorization and georecognition have been carried out employing a well-known geographical image repository: results are actually very promising, opening new interesting challenges and applications in this research field.

The bag of words approach for retrieval and categorization of 3D objects

In this paper, we propose a novel framework for 3D object retrieval and categorization. The object is modeled in terms of its subparts as an histogram of 3D visual word occurrences. We introduce an effective method for hierarchical 3D object segmentation driven by the minima rule that combines spectral clustering—for the selection of seed-regions—with region growing based on fast marching. Descriptors attached to the regions allow the definition of the visual words. After coding of each object according to the Bag-of-Words paradigm, retrieval can be performed by matching with a suitable kernel, or categorization by learning a Support Vector Machine. Several examples on the Aim@Shape watertight dataset and on the Tosca dataset demonstrate the versatility of the proposed method in working with either 3D objects with articulated shape changes or partially occluded or compound objects. Results are encouraging as shown by the comparison with other methods for each of the analyzed scenarios.

Registration of multiple acoustic range views for underwater scene reconstruction

This paper proposes a technique for the three-dimensional reconstruction of an underwater environment from multiple acoustic range views acquired by a remotely operated vehicle. The problem is made challenging by the very noisy nature of the data. the low resolution, and the narrow field of view. Our main contribution is a new global registration technique to distribute registration errors evenly across all views. Our approach does not use data points after the first pairwise registration, for it works only on the transformations. Therefore. it is fast and occupies only a small amount of memory. Experimental results suggest the global registration technique is effective in equalizing the error. Moreover, we introduce a statistically sound thresholding (the X84 rejection rule) to improve ICP robustness against noise and nonoverlapping data.

Free Energy Score Spaces: Using Generative Information in Discriminative Classifiers

A score function induced by a generative model of the data can provide a feature vector of a fixed dimension for each data sample. Data samples themselves may be of differing lengths (e.g., speech segments or other sequential data), but as a score function is based on the properties of the data generation process, it produces a fixed-length vector in a highly informative space, typically referred to as “score space.” Discriminative classifiers have been shown to achieve higher performances in appropriately chosen score spaces with respect to what is achievable by either the corresponding generative likelihood-based classifiers or the discriminative classifiers using standard feature extractors. In this paper, we present a novel score space that exploits the free energy associated with a generative model. The resulting free energy score space (FESS) takes into account the latent structure of the data at various levels and can be shown to lead to classification performance that at least matches the performance of the free energy classifier based on the same generative model and the same factorization of the posterior. We also show that in several typical computer vision and computational biology applications the classifiers optimized in FESS outperform the corresponding pure generative approaches, as well as a number of previous approaches combining discriminating and generative models.