Mukta Prasad

Hough transform and 3D SURF for robust three dimensional classification

Most methods for the recognition of shape classes from 3D datasets focus on classifying clean, often manually generated models. However, 3D shapes obtained through acquisition techniques such as Structure-from-Motion or LIDAR scanning are noisy, clutter and holes. In that case global shape features--still dominating the 3D shape class recognition literature--are less appropriate. Inspired by 2D methods, recently researchers have started to work with local features. In keeping with this strand, we propose a new robust 3D shape classification method. It contains two main contributions. First, we extend a robust 2D feature descriptor, SURF, to be used in the context of 3D shapes. Second, we show how 3D shape class recognition can be improved by probabilistic Hough transform based methods, already popular in 2D. Through our experiments on partial shape retrieval, we show the power of the proposed 3D features. Their combination with the Hough transform yields superior results for class recognition on standard datasets. The potential for the applicability of such a method in classifying 3D obtained from Structure-from-Motion methods is promising, as we show in some initial experiments.

Transforming image completion

Image completion is an important photo-editing task which involves synthetically filling a hole in the image such that the image still appears natural. State-of-the-art image completion methods work by searching for patches in the image that fit well in the hole region. Our key insight is that image patches remain natural under a variety of transformations (such as scale, rotation and brightness change), and it is important to exploit this. We propose and investigate the use of different optimisation methods to search for the best patches and their respective transformations for producing consistent, improved completions. Experiments on a number of challenging problem instances demonstrate that our methods outperform state-of-the-art techniques.

Orientation invariant 3D object classification using hough transform based methods

In comparison to the 2D case, object class recognition in 3D is a much less researched area. However, with the advent of affordable 3D acquisition technology and the growing popularity of 3D content, its relevance is steadily increasing. Just as in the 2D case, 3D data is often partial, noisy and without prior segmentation. Moreover, the object is rarely observed in a canonical frame of reference with respect to orientation (or scale). We propose a method, using Hough-voting for local 3D features, which is orientation (and scale) invariant.

Fast and Controllable 3D Modelling From Silhouettes

We show how a 3D model of a complex curved object can be easily extracted from a single 2D image. A userdefined silhouette is the key input; and we show that finding the smoothest 3D surface which projects exactly to this silhouette can be expressed as a quadratic optimization, a result which has not previously appeared in the large literature on the shape-from-silhouette problem. For simple models, this process can immediately yield a usable 3D model; but for more complex geometries the user will wish to further shape the surface. We show that a variety of editing operations—which can be defined either in the image or in 3D—can also be expressed as linear constraints on the 3D shape parameters. We extend the system to fit higher genus surfaces. Our method has several advantages over the system of Zhanget al. [ZDPSS01] and over systems such asSKETCH and Teddy.

Ensemble partitioning for unsupervised image categorization

While the quality of object recognition systems can strongly benefit from more data, human annotation and labeling can hardly keep pace. This motivates the usage of autonomous and unsupervised learning methods. In this paper, we present a simple, yet effective method for unsupervised image categorization, which relies on discriminative learners. Since automatically obtaining error-free labeled training data for the learners is infeasible, we propose the concept of weak training (WT) set. WT sets have various deficiencies, but still carry useful information. Training on a single WT set cannot result in good performance, thus we design a random walk sampling scheme to create a series of diverse WT sets. This naturally allows our categorization learning to leverage ensemble learning techniques. In particular, for each WT set, we train a max-margin classifier to further partition the whole dataset to be categorized. By doing so, each WT set leads to a base partitioning of the dataset and all the base partitionings are combined into an ensemble proximity matrix. The final categorization is completed by feeding this proximity matrix into a spectral clustering algorithm. Experiments on a variety of challenging datasets show that our method outperforms competing methods by a considerable margin.

Learning class-specific edges for object detection and segmentation

Recent research into recognizing object classes (such as humans, cows and hands) has made use of edge features to hypothesize and localize class instances. However, for the most part, these edge-based methods operate solely on the geometric shape of edges, treating them equally and ignoring the fact that for certain object classes, the appearance of the object on the “inside” of the edge may provide valuable recognition cues. We show how, for such object classes, small regions around edges can be used to classify the edge into object or non-object. This classifier may then be used to prune edges which are not relevant to the object class, and thereby improve the performance of subsequent processing. We demonstrate learning class specific edges for a number of object classes — oranges, bananas and bottles — under challenging scale and illumination variation. Because class-specific edge classification provides a low-level analysis of the image it may be integrated into any edge-based recognition strategy without significant change in the high-level algorithms. We illustrate its application to two algorithms: (i) chamfer matching for object detection, and (ii) modulating contrast terms in MRF based object-specific segmentation. We show that performance of both algorithms (matching and segmentation) is considerably improved by the class-specific edge labelling.

Finding nemo: Deformable object class modelling using curve matching

An image search for “clownfish” yields many photos of clownfish, each of a different individual of a different 3D shape in a different pose. Yet, to the human observer, this set of images contains enough information to infer the underlying 3D deformable object class. Our goal is to recover such a deformable object class model directly from unordered images. For classes where feature-point correspondences can be found, this is a straightforward extension of non-rigid factorization, yielding a set of 3D basis shapes to explain the 2D data. However, when each image is of a different object instance, surface texture is generally unique to each individual, and does not give rise to usable image point correspondences. We overcome this sparsity using curve correspondences (crease-edge silhouettes or class-specific internal texture edges). Even rigid contour reconstruction is difficult due to the lack of reliable correspondences. We incorporate correspondence variation into the optimization, thereby extending contour-based reconstruction techniques to deformable object modelling. The notion of correspondence is extended to include mappings between 2D image curves and corresponding parts of the desired 3D object surface. Combined with class-specific priors, our method enables effective de-formable class reconstruction from unordered images, despite significant occlusion and the scarcity of shared 2D image features.

Is There a Procedural Logic to Architecture

Urban models are key to navigation, architecture and entertainment. Apart from visualizing facades, a number of tedious tasks remain largely manual (e.g. compression, generating new facade designs and structurally comparing facades for classification, retrieval and clustering). We propose a novel procedural modelling method to automatically learn a grammar from a set of facades, generate new facade instances and compare facades. To deal with the difficulty of grammatical inference, we reformulate the problem. Instead of inferring a compromising, one-size-fits-all, single grammar for all tasks, we infer a model whose successive refinements are production rules tailored for each task. We demonstrate our automatic rule inference on datasets of two different architectural styles. Our method supercedes manual expert work and cuts the time required to build a procedural model of a facade from several days to a few milliseconds.

Scene Cut: Class-Specific Object Detection and Segmentation in 3D Scenes

In this paper we present a method to combine the detection and segmentation of object categories from 3D scenes. In the process, we combine the top-down cues available from object detection technique of Implicit Shape Models and the bottom-up power of Markov Random Fields for the purpose of segmentation. While such approaches have been tried for the 2D image problem domain before, this is the first application of such a method in 3D. 3D scene understanding is prone to many problems different from 2D owing to problems from noise, lack of distinctive high-frequency feature information, mesh parametrization problems etc. Our method enables us to localize objects of interest for more purposeful meshing and subsequent scene understanding.

Learning domain knowledge for façade labelling

This paper presents an approach to address the problem of image facade labelling. In the architectural literature, domain knowledge is usually expressed geometrically in the final design, so facade labelling should on the one hand conform to visual evidence, and on the other hand to the architectural principles – how individual assets (e.g. doors, windows) interact with each other to form a facade as a whole. To this end, we first propose a recursive splitting method to segment facades into a bunch of tiles for semantic recognition. The segmentation improves the processing speed, guides visual recognition on suitable scales and renders the extraction of architectural principles easy. Given a set of segmented training facades with their label maps, we then identify a set of meta-features to capture both the visual evidence and the architectural principles. The features are used to train our facade labelling model. In the test stage, the features are extracted from segmented facades and the inferred label maps. The following three steps are iterated until the optimal labelling is reached: 1) proposing modifications to the current labelling; 2) extracting new features for the proposed labelling; 3) feeding the new features to the labelling model to decide whether to accept the modifications. In experiments, we evaluated our method on the ECP facade dataset and achieved higher precision than the state-of-the-art at both the pixel level and the structural level.