Roderic Collins

A common set of perceptual observables for grouping, figure-ground discrimination, and texture classification

We present a complete set of perceptual observables that provides a unified image description for grouping, figure-ground separation, and texture analysis. Although much progress has been made recently in treating contours and texture simultaneously for image segmentation and grouping, current approaches rely on different models for contours, regions, and texture such as one-dimensional intensity discontinuities for contours and filter bank responses for texture. This results in expensive computation that arbitrates between these disparate representations at each pixel. In our approach, salient image content such as contours, regions, and texture are represented in a common, low-level framework of image observables. We model the image as a partition of surfaces bounded by intensity discontinuities and derive perceptual measures as relations between neighboring surfaces. This enables us to extend the traditional Gestalt measures based on local edge geometry and contrast to region-based measures that jointly exploit large scale image topology, photometry, and geometry. These measures provide a natural basis for grouping on multidimensional similarity criteria and texture is directly derived as relational properties on local region neighborhoods. The viability of our model is demonstrated by applying the common observables to texture recognition, figure-ground separation, and generic image segmentation.

Recognition and Segmentation of Scene Content using Region-Based Classification

We present a novel method for joint segmentation and pixelwise classification of images, classifying each pixel in the image into one of a set of broad categories. We propose a 2-step approach for this problem, first estimating image structure through dense region segmentation, which provides initial spatial grouping (superpixels), then performing recognition by classifying each superpixel according to its features. Two types of region features are investigated: perceptual grouping features derived from neighborhood relations in the superpixel graph, and a histogram of pixel textons within the superpixel. Region classification is performed by boosting for perceptual features and histogram matching for texton features. We also introduce a novel extension of multi-class boosting: MAP estimation in the space of classifier ensemble outputs. Extensive results on aerial imagery are presented using a label vocabulary of trees, roads, vehicles, grass, shadows, and buildings. We evaluate the two methods across the categories, and compare them to the standard approach of classifying image blocks without prior segmentation. In our experiments perceptual features using multi-class boosting provide the best performance

Object Boundary Detection in Images using a Semantic Ontology

We present a novel method for detecting the boundaries between objects in images that uses a large, hierarchical, semantic ontology - WordNet. The semantic object hierarchy in WordNet grounds this ill-posed segmentation problem, so that true boundaries are defined as edges between instances of different classes, and all other edges are clutter. To avoid fully classifying each pixel, which is very difficult in generic images, we evaluate the semantic similarity of the two regions bounding each edge in an initial oversegmentation. Semantic similarity is computed using WordNet enhanced with appearance information, and is largely orthogonal to visual similarity. Hence two regions with very similar visual attributes, but from different categories, can have a large semantic distance and therefore evidence of a strong boundary between them, and vice versa. The ontology is trained with images from the UC Berkeley image segmentation benchmark, extended with manual labeling of the semantic content of each image segment. Results on boundary detection against the benchmark images show that semantic similarity computed through WordNet can significantly improve boundary detection compared to generic segmentation.

Classification of 3D macro texture using perceptual observables

A new method for analyzing macro texture using perceptual observables is presented. The typical geometric Gestalt grouping criteria such as proximity and parallelism are extended with descriptive measures of topology and photometry enabled by region neighborhood analysis. It is proposed that these perceptual measures provide a common description of image content encompassing both macro texture and perceptual grouping. This theory enables a new algorithm for macro texture classification that is invariant to rotation, and robust against very large changes in illumination, viewpoint and scale. The classification process also provides a method to determine which perceptual attributes are the most relevant for discriminating between various textures and objects.

Using video for recovering texture

Existing approaches to characterizing image texture usually rely on computing a local response to a bank of correlation filters, such as derivatives of a Gaussian, in one image. Recently, significant progress has been made in characterizing a single texture under varying viewpoint and illumination conditions, leading to the bi-directional texture function that describes the smooth variation of filter responses as a function of viewpoint and illumination. However, this technique does not attempt to exploit the redundancy of multiple images; each image is treated independently. In video data, close correspondences between frames enable a new form of texture analysis that incorporates local 3D structure as well as intensity variation. We exploit this relationship to characterize texture with significant 3D structure, such as foliage, across a range of viewpoints. This paper presents a general overview of these ideas and preliminary results.