Davide Boscaini

Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs

Deep learning has achieved a remarkable performance breakthrough in several fields, most notably in speech recognition, natural language processing, and computer vision. In particular, convolutional neural network (CNN) architectures currently produce state-of-the-art performance on a variety of image analysis tasks such as object detection and recognition. Most of deep learning research has so far focused on dealing with 1D, 2D, or 3D Euclidean-structured data such as acoustic signals, images, or videos. Recently, there has been an increasing interest in geometric deep learning, attempting to generalize deep learning methods to non-Euclidean structured data such as graphs and manifolds, with a variety of applications from the domains of network analysis, computational social science, or computer graphics. In this paper, we propose a unified framework allowing to generalize CNN architectures to non-Euclidean domains (graphs and manifolds) and learn local, stationary, and compositional task-specific features. We show that various non-Euclidean CNN methods previously proposed in the literature can be considered as particular instances of our framework. We test the proposed method on standard tasks from the realms of image-, graph-and 3D shape analysis and show that it consistently outperforms previous approaches.

Learning class-specific descriptors for deformable shapes using localized spectral convolutional networks

In this paper, we propose a generalization of convolutional neural networks (CNN) to non‐Euclidean domains for the analysis of deformable shapes. Our construction is based on localized frequency analysis (a generalization of the windowed Fourier transform to manifolds) that is used to extract the local behavior of some dense intrinsic descriptor, roughly acting as an analogy to patches in images. The resulting local frequency representations are then passed through a bank of filters whose coefficient are determined by a learning procedure minimizing a task‐specific cost. Our approach generalizes several previous methods such as HKS, WKS, spectral CNN, and GPS embeddings. Experimental results show that the proposed approach allows learning class‐specific shape descriptors significantly outperforming recent state‐of‐the‐art methods on standard benchmarks.

Anisotropic Diffusion Descriptors

Spectral methods have recently gained popularity in many domains of computer graphics and geometry processing, especially shape processing, computation of shape descriptors, distances, and correspondence. Spectral geometric structures are intrinsic and thus invariant to isometric deformations, are efficiently computed, and can be constructed on shapes in different representations. A notable drawback of these constructions, however, is that they are isotropic, i.e., insensitive to direction. In this paper, we show how to construct direction‐sensitive spectral feature descriptors using anisotropic diffusion on meshes and point clouds. The core of our construction are directed local kernels acting similarly to steerable filters, which are learned in a task‐specific manner. Remarkably, while being intrinsic, our descriptors allow to disambiguate reflection symmetries. We show the application of anisotropic descriptors for problems of shape correspondence on meshes and point clouds, achieving results significantly better than state‐of‐the‐art methods.

Shape-from-Operator: Recovering Shapes from Intrinsic Operators

We formulate the problem of shape‐from‐operator (SfO), recovering an embedding of a mesh from intrinsic operators defined through the discrete metric (edge lengths). Particularly interesting instances of our SfO problem include: shape‐from‐Laplacian, allowing to transfer style between shapes; shape‐from‐difference operator, used to synthesize shape analogies; and shape‐from‐eigenvectors, allowing to generate ‘intrinsic averages’ of shape collections. Numerically, we approach the SfO problem by splitting it into two optimization sub‐problems: metric‐from‐operator (reconstruction of the discrete metric from the intrinsic operator) and embedding‐from‐metric (finding a shape embedding that would realize a given metric, a setting of the multidimensional scaling problem). We study numerical properties of our problem, exemplify it on several applications, and discuss its imitations.

A sparse coding approach for local-to-global 3D shape description

The definition of reliable shape descriptors is an essential topic for 3D object retrieval. In general, two main approaches are considered: global, and local. Global approaches are effective in describing the whole object, while local ones are more suitable to characterize small parts of the shape. Recently some strategies to combine these two approaches have been proposed which are mainly concentrated to the so-called bag of words paradigm. With this paper we address this problem and propose an alternative strategy that goes beyond the bag of word approach. In particular, a sparse coding technique is exploited for the 3D domain: a set of local shape descriptors are collected from the shape, and then a dictionary is trained as generative model. In this fashion the dictionary is used as global shape descriptor for shape retrieval purposes. Several experiments are performed on standard databases in order to evaluate the proposed method in challenging situations like the case of ‘SHREC 2011: robustness benchmark’ where strong shape transformations are included, and the case of ‘SHREC 2007: partial matching track’ where composite models are considered in the query phase. A drastic improvement of the proposed method is observed by showing that sparse coding approach is particularly suitable for local-to-global description and outperforms other approaches such as the bag of words.

Local signature quantization by sparse coding

In 3D object retrieval it is very important to define reliable shape descriptors, which compactly characterize geometric properties of the underlying surface. To this aim two main approaches are considered: global, and local ones. Global approaches are effective in describing the whole object, while local ones are more suitable to characterize small parts of the shape. Some strategies to combine these two approaches have been proposed recently but still no consolidate work is available in this field. With this paper we address this problem and propose a new method based on sparse coding techniques. A set of local shape descriptors are collected from the shape. Then a dictionary is trained as generative model. In this fashion the dictionary is used as global shape descriptor for shape retrieval purposes. Preliminary experiments are performed on a standard dataset by showing a drastic improvement of the proposed method in comparison with well known local-to-global and global approaches.

Geometric Deep Learning for Shape Analysis

The past decade in computer vision research has witnessed the re-emergence of artificial neural networks (ANN), and in particular convolutional neural network (CNN) techniques, allowing to learn powerful feature representations from large collections of data. Nowadays these techniques are better known under the umbrella term deep learning and have achieved a breakthrough in performance in a wide range of image analysis applications such as image classification, segmentation, and annotation. Nevertheless, when attempting to apply deep learning paradigms to 3D shapes one has to face fundamental differences between images and geometric objects. The main difference between images and 3D shapes is the non-Euclidean nature of the latter. This implies that basic operations, such as linear combination or convolution, that are taken for granted in the Euclidean case, are not even well defined on non-Euclidean domains. This happens to be the major obstacle that so far has precluded the successful application of deep learning methods on non-Euclidean geometric data. The goal of this thesis is to overcome this obstacle by extending deep learning tecniques (including, but not limiting to CNNs) to non-Euclidean domains. We present different approaches providing such extension and test their effectiveness in the context of shape similarity and correspondence applications. The proposed approaches are evaluated on several challenging experiments, achieving state-ofthe-art results significantly outperforming other methods. To the best of our knowledge, this thesis presents different original contributions. First, this work pioneers the generalization of CNNs to discrete manifolds. Second, it provides an alternative formulation of the spectral convolution operation in terms of the windowed Fourier transform to overcome the drawbacks of the Fourier one. Third, it introduces a spatial domain formulation of convolution operation using patch operators and several ways of their construction (geodesic, anisotropic diffusion, mixture of Gaussians). Fourth, at the moment of publication the proposed approaches achieved state-of-the-art results in different computer graphics and vision applications such as shape descriptors and correspondence.