Hamid Laga

Spherical Wavelet Descriptors for Content-based 3D Model Retrieval

The description of 3D shapes with features that possess descriptive power and invariant under similarity transformations is one of the most challenging issues in content based 3D model retrieval. Spherical harmonics-based descriptors have been proposed for obtaining rotation invariant representations. However, spherical harmonic analysis is based on latitude-longitude parameterization of a sphere which has singularities at each pole. Consequently, features near the two poles are over represented while features at the equator are under-sampled, and variations of the north pole affects significantly the shape function. In this paper we discuss these issues and propose the usage of spherical wavelet transform as a tool for the analysis of 3D shapes represented by functions on the unit sphere. We introduce three new descriptors extracted from the wavelet coefficients, namely: (1) a subset of the spherical wavelet coefficients, (2) the L1 and, (3) the L2 energies of the spherical wavelet sub-bands. The advantage of this tool is three fold; first, it takes into account feature localization and local orientations. Second, the energies of the wavelet transform are rotation invariant. Third, shape features are uniformly represented which makes the descriptors more efficient. Spherical wavelet descriptors are natural extension of 3D Zernike moments and spherical harmonics. We evaluate, on the Princeton shape benchmark, the proposed descriptors regarding computational aspects and shape retrieval performance

Covariance Descriptors for 3D Shape Matching and Retrieval

Several descriptors have been proposed in the past for 3D shape analysis, yet none of them achieves best performance on all shape classes. In this paper we propose a novel method for 3D shape analysis using the covariance matrices of the descriptors rather than the descriptors themselves. Covariance matrices enable efficient fusion of different types of features and modalities. They capture, using the same representation, not only the geometric and the spatial properties of a shape region but also the correlation of these properties within the region. Covariance matrices, however, lie on the manifold of Symmetric Positive Definite (SPD) tensors, a special type of Riemannian manifolds, which makes comparison and clustering of such matrices challenging. In this paper we study covariance matrices in their native space and make use of geodesic distances on the manifold as a dissimilarity measure. We demonstrate the performance of this metric on 3D face matching and recognition tasks. We then generalize the Bag of Features paradigm, originally designed in Euclidean spaces, to the Riemannian manifold of SPD matrices. We propose a new clustering procedure that takes into account the geometry of the Riemannian manifold. We evaluate the performance of the proposed Bag of Covariance Matrices framework on 3D shape matching and retrieval applications and demonstrate its superiority compared to descriptor-based techniques.

Geometry and context for semantic correspondences and functionality recognition in man-made 3D shapes

We address the problem of automatic recognition of functional parts of man-made 3D shapes in the presence of significant geometric and topological variations. We observe that under such challenging circumstances, the context of a part within a 3D shape provides important cues for learning the semantics of shapes. We propose to model the context as structural relationships between shape parts and use them, in addition to part geometry, as cues for functionality recognition. We represent a 3D shape as a graph interconnecting parts that share some spatial relationships. We model the context of a shape part as walks in the graph. Similarity between shape parts can then be defined as the similarity between their contexts, which in turn can be efficiently computed using graph kernels. This formulation enables us to: (1) find part-wise semantic correspondences between 3D shapes in a nonsupervised manner and without relying on user-specified textual tags, and (2) design classifiers that learn in a supervised manner the functionality of the shape components. We specifically show that the performance of the proposed context-aware similarity measure in finding part-wise correspondences outperforms geometry-only-based techniques and that contextual analysis is effective in dealing with shapes exhibiting large geometric and topological variations.

Landmark-Guided Elastic Shape Analysis of Spherically-Parameterized Surfaces

We argue that full surface correspondence (registration) and optimal deformations (geodesics) are two related problems and propose a framework that solves them simultaneously. We build on the Riemannian shape analysis of anatomical and star‐shaped surfaces of Kurtek et al. and focus on articulated complex shapes that undergo elastic deformations and that may contain missing parts. Our core contribution is the re‐formulation of Kurtek et al.s approach as a constrained optimization over all possible re‐parameterizations of the surfaces, using a sparse set of corresponding landmarks. We introduce a landmark‐constrained basis, which we use to numerically solve this optimization and therefore establish full surface registration and geodesic deformation between two surfaces. The length of the geodesic provides a measure of dissimilarity between surfaces. The advantages of this approach are: (1) simultaneous computation of full correspondence and geodesic between two surfaces, given a sparse set of matching landmarks (2) ability to handle more comprehensive deformations than nearly isometric, and (3) the geodesics and the geodesic lengths can be further used for symmetrizing 3D shapes and for computing their statistical averages. We validate the framework on challenging cases of large isometric and elastic deformations, and on surfaces with missing parts. We also provide multiple examples of averaging and symmetrizing 3D models.

Semantics-driven approach for automatic selection of best views of 3D shapes

We introduce a new framework for the automatic selection of the best views of 3D models. The approach is based on the assumption that models belonging to the same class of shapes share the same salient features that discriminate them from the models of other classes. The main issue is learning these features. We propose a datadriven approach where the best view selection problem is formulated as a classification and feature selection problem; First a 3D model is described with a set of view-based descriptors, each one computed from a different viewpoint. Then a classifier is trained, in a supervised manner, on a collection of 3D models belonging to several shape categories. The classifier learns the set of 2D views that maximize the similarity between shapes of the same class and also the views that discriminate shapes of different classes. Our experiments using the LightField (LFD) descriptors and the Princeton Shape Benchmark demonstrate the performance of the approach and its suitability for classification and online visual browsing of 3D data collections.

A Riemannian Elastic Metric for Shape-Based Plant Leaf Classification

The shapes of plant leaves are of great importance to plant biologists and botanists, as they can help to distinguish plant species and measure their health. In this paper, we study the performance of the Squared Root Velocity Function (SRVF) representation of closed planar curves in the analysis of plant-leaf shapes. We show that it provides a joint framework for computing geodesics (registration) and similarities between plant leaves, which we use for their automatic classification. We evaluate its performance using standard databases and show that it outperforms significantly the state-of-the-art descriptor-based techniques. Additionally, we show that it enables the computation of shape statistics, such as the average shape of a leaf population and its principal directions of variation, suggesting that the representation is suitable for building generative models of plant- leaf shapes.

Landmark-free statistical analysis of the shape of plant leaves

The shapes of plant leaves are important features to biologists, as they can help in distinguishing plant species, measuring their health, analyzing their growth patterns, and understanding relations between various species. Most of the methods that have been developed in the past focus on comparing the shape of individual leaves using either descriptors or finite sets of landmarks. However, descriptor-based representations are not invertible and thus it is often hard to map descriptor variability into shape variability. On the other hand, landmark-based techniques require automatic detection and registration of the landmarks, which is very challenging in the case of plant leaves that exhibit high variability within and across species. In this paper, we propose a statistical model based on the Squared Root Velocity Function (SRVF) representation and the Riemannian elastic metric of Srivastava et al. (2011) to model the observed continuous variability in the shape of plant leaves. We treat plant species as random variables on a non-linear shape manifold and thus statistical summaries, such as means and covariances, can be computed. One can then study the principal modes of variations and characterize the observed shapes using probability density models, such as Gaussians or Mixture of Gaussians. We demonstrate the usage of such statistical model for (1) efficient classification of individual leaves, (2) the exploration of the space of plant leaf shapes, which is important in the study of population-specific variations, and (3) comparing entire plant species, which is fundamental to the study of evolutionary relationships in plants. Our approach does not require descriptors or landmarks but automatically solves for the optimal registration that aligns a pair of shapes. We evaluate the performance of the proposed framework on publicly available benchmarks such as the Flavia, the Swedish, and the ImageCLEF2011 plant leaf datasets.

Compact vectors of locally aggregated tensors for 3D shape retrieval

During the last decade, a significant attention has been paid, by the computer vision and the computer graphics communities, to three dimensional (3D) object retrieval. Shape retrieval methods can be divided into three main steps: the shape descriptors extraction, the shape signatures and their associated similarity measures, and the machine learning relevance functions. While the first and the last points have vastly been addressed in recent years, in this paper, we focus on the second point; presenting a new 3D object retrieval method using a new coding/pooling technique and powerful 3D shape descriptors extracted from 2D views. For a given 3D shape, the approach extracts a very large and dense set of local descriptors. From these descriptors, we build a new shape signature by aggregating tensor products of visual descriptors. The similarity between 3D models can then be efficiently computed with a simple dot product. We further improve the compactness and discrimination power of the descriptor using local Principal Component Analysis on each cluster of descriptors. Experiments on the SHREC 2012 and the McGill benchmarks show that our approach outperforms the state-of-the-art techniques, including other BoF methods, both in compactness of the representation and in the retrieval performance.

IR2s: interactive real photo to Sumi-e

We propose an interactive sketch-based system for rendering oriental brush strokes on complex shapes. We introduce a contour-driven approach; the user inputs contours to represent complex shapes, the system estimates automatically the optimal trajectory of the brush, and then renders them into oriental ink paintings. Unlike previous work where the brush trajectory is explicitly provided as input, we automatically estimate this trajectory from the outline of the shapes to paint using a three-stages algorithm; first complex shapes are decomposed into elementary shapes that can be rendered with a single brush stroke. Second, we formulate the optimal brush trajectory estimation as the minimization of an energy function that measures the quality of the trajectory constrained by the variation along a stroke of the painting process parameters, such as the footprint position, size, orientation, and angular velocity. Finally, the estimated trajectories are rendered into brush strokes by mapping footprint textures scanned from real images. We combine the proposed framework with an interactive segmentation in order to convert real images into Oriental ink paintings. Experiments on complex shapes show that the proposed contour-based approach produces a large variety of strokes compared to trajectory-based approaches. It is particularly suitable for converting real images into Oriental ink paintings with minimum interaction.

Covariance-Based Descriptors for Efficient 3D Shape Matching, Retrieval, and Classification

State-of-the-art 3D shape classification and retrieval algorithms, hereinafter referred to as shape analysis, are often based on comparing signatures or descriptors that capture the main geometric and topological properties of 3D objects. None of the existing descriptors, however, achieve best performance on all shape classes. In this article, we explore, for the first time, the usage of covariance matrices of descriptors, instead of the descriptors themselves, in 3D shape analysis. Unlike histogram -based techniques, covariance-based 3D shape analysis enables the fusion and encoding of different types of features and modalities into a compact representation. Covariance matrices, however, are elements of the non-linear manifold of symmetric positive definite (SPD) matrices and thus \BBL2 metrics are not suitable for their comparison and clustering. In this article, we study geodesic distances on the Riemannian manifold of SPD matrices and use them as metrics for 3D shape matching and recognition. We then: (1) introduce the concepts of bag of covariance (BoC) matrices and spatially-sensitive BoC as a generalization to the Riemannian manifold of SPD matrices of the traditional bag of features framework, and (2) generalize the standard kernel methods for supervised classification of 3D shapes to the space of covariance matrices. We evaluate the performance of the proposed BoC matrices framework and covariance -based kernel methods and demonstrate their superiority compared to their descriptor-based counterparts in various 3D shape matching, retrieval, and classification setups.