Adrian Ion

Image segmentation by figure-ground composition into maximal cliques

We propose a mid-level statistical model for image segmentation that composes multiple figure-ground hypotheses (FG) obtained by applying constraints at different locations and scales, into larger interpretations (tilings) of the entire image. Inference is cast as optimization over sets of maximal cliques sampled from a graph connecting all non-overlapping figure-ground segment hypotheses. Potential functions over cliques combine unary, Gestalt-based figure qualities, and pairwise compatibilities among spatially neighboring segments, constrained by T-junctions and the boundary interface statistics of real scenes. Learning the model parameters is based on maximum likelihood, alternating between sampling image tilings and optimizing their potential function parameters. State of the art results are reported on the Berkeley and Stanford segmentation datasets, as well as VOC2009, where a 28% improvement was achieved.

Video Object Segmentation by Salient Segment Chain Composition

We present a model for video segmentation, applicable to RGB (and if available RGB-D) information that constructs multiple plausible partitions corresponding to the static and the moving objects in the scene: i) we generate multiple figure-ground segmentations, in each frame, parametrically, based on boundary and optical flow cues, then track, link and refine the salient segment chains corresponding to the different objects, over time, using long-range temporal constraints, ii) a video partition is obtained by composing segment chains into consistent tilings, where the different individual object chains explain the video and do not overlap. Saliency metrics based on figural and motion cues, as well as measures learned from human eye movements are exploited, with substantial gain, at the level of segment generation and chain construction, in order to produce compact sets of hypotheses which correctly reflect the qualities of the different configurations. The model makes it possible to compute multiple hypotheses over both individual object segmentations tracked over time, and for complete video partitions. We report quantitative, state of the art results in the SegTrack single object benchmark, and promising qualitative and quantitative results in clips filming multiple static and moving objects collected from Hollywood movies and from the MIT dataset.

Multiresolution Image Segmentations in Graph Pyramids

”How do we bridge the representational gap between image features and coarse model features?” is the question asked by the authors of [47] when referring to several contemporary research issues. They identify the one-to-one correspondence between salient image features (pixels, edges, corners,...) and salient model features (generalized cylinders, polyhedrons, invariant models,...) as a limiting assumption that makes prototypical or generic object recognition impossible. They suggested to bridge and not to eliminate the representational gap, as it is done in the computer vision community for quite long, and to focus efforts on: i) region segmentation, ii) perceptual grouping, and iii) image abstraction. Let us take these goals as a guideline to consider multiresolution representations under the special viewpoint of segmentation and grouping. In [34] multiresolution representation is considered under the abstraction viewpoint. Wertheimer [51] has formulated the importance of wholes (Ganzen) and not of its individual elements and introduced the importance of perceptual grouping and organization in visual perception. Regions as aggregations of primitive pixels play an extremely important role in nearly every image analysis task. Their internal properties (color, texture, shape, ...) help to identify them, and their external relations (adjacency, inclusion, similarity of properties) are used to build groups of regions having a particular meaning in a more abstract context. The union of regions forming the group is again a region with both internal and external properties and relations. Low-level cue image segmentation can not and should not produce a complete final ’good’ segmentation, because there is no general ’good’ segmentation. Without prior knowledge, segmentation based on low-level cues will not be able to extract semantics in generic images. Using some similarity measures, the segmentation process results in ‘homogeneity’ regions with respect to the low-level cues. Problems emerge because i) homogeneity of low-level cues will not map to the semantics [28] and ii) the degree of homogeneity of a region is in general quantified by threshold(s) for a given measure [12]. Even though segmentation methods (including ours) that do not take the context of the image into consideration can not produce a ’good’ segmentation, they can be valuable tools in image analysis in the same sense as efficient edge detectors are. Note that efficient edge detectors do not consider the context of the image, too. Thus, the low-level coherence of brightness, color, texture or motion attributes should be used to sequentially come up with hierarchical partitions [46]. Mid and high level knowledge can be used to either confirm these groups or select some further attention. A wide range of computational vision problems could make use of segmented images, were such segmentation rely on efficient computation, e.g. motion estimation requires an appropriate region of support for finding correspondences; higher-level problems such as recognition and image indexing can also make use of segmentation results in the problem of matching. It is important for a grouping method to have the following properties [10]:

Matching 2D and 3D articulated shapes using the eccentricity transform

This paper presents a novel method for 2D and 3D shape matching that is insensitive to articulation. It uses the eccentricity transform, which is based on the computation of geodesic distances. Geodesic distances computed over a 2D or 3D shape are articulation insensitive. The eccentricity transform considers the length of the longest geodesics. Histograms of the eccentricity transform characterize the compactness of a shape, in a way insensitive to rotation, scaling, and articulation. To characterize the structure of a shape, a histogram of the connected components of the level-sets of the transform is used. These two histograms make up a highly compact descriptor and the resulting method for shape matching is straightforward. Experimental results on established 2D and 3D benchmarks show results similar to more complex state of the art methods, especially when considering articulation. The connection between the geometrical modification of a shape and the corresponding impact on its histogram representation is explained. The influence of the number of bins in the two histograms and the respective importance of each histogram is studied in detail.

Directly computing the generators of image homology using graph pyramids

We introduce a method for computing homology groups and their generators of a 2D image, using a hierarchical structure, i.e. irregular graph pyramid. Starting from an image, a hierarchy of the image is built by two operations that preserve homology of each region. Instead of computing homology generators in the base where the number of entities (cells) is large, we first reduce the number of cells by a graph pyramid. Then homology generators are computed efficiently on the top level of the pyramid, since the number of cells is small. A top down process is then used to deduce homology generators in any level of the pyramid, including the base level, i.e. the initial image. The produced generators fit on the object boundaries. A unique set of generators called the minimal set, is defined and its computation is discussed. We show that the new method produces valid homology generators and present some experimental results.

Computing homology group generators of images using irregular graph pyramids

We introduce a method for computing homology groups and their generators of a 2D image, using a hierarchical structure i.e. irregular graph pyramid. Starting from an image, a hierarchy of the image is built, by two operations that preserve homology of each region. Instead of computing homology generators in the base where the number of entities (cells) is large, we first reduce the number of cells by a graph pyramid. Then homology generators are computed efficiently on the top level of the pyramid, since the number of cells is small, and a top down process is then used to deduce homology generators in any level of the pyramid, including the base level i.e. the initial image. We show that the new method produces valid homology generators and present some experimental results.

3D shape matching by geodesic eccentricity

This paper makes use of the continuous eccentricity transform to perform 3D shape matching. The eccentricity transform has already been proved useful in a discrete graph-theoretic setting and has been applied to 2D shape matching. We show how these ideas extend to higher dimensions. The eccentricity transform is used to compute descriptors for 3D shapes. These descriptors are defined as histograms of the eccentricity transform and are naturally invariant to Euclidean motion and articulation. They show promising results for shape discrimination.

The eccentricity transform (of a digital shape)

Eccentricity measures the shortest length of the paths from a given vertex v to reach any other vertex w of a connected graph Computed for every vertex v it transforms the connectivity structure of the graph into a set of values For a connected region of a digital image it is defined through its neighbourhood graph and the given metric This transform assigns to each element of a region a value that depends on its location inside the region and the regions shape The definition and several properties are given Presented experimental results verify its robustness against noise, and its increased stability compared to the distance transform Future work will include using it for shape decomposition, representation, and matching.

Evaluating minimum spanning tree based segmentation algorithms

Two segmentation methods based on the minimum spanning tree principle are evaluated with respect to each other. The hierarchical minimum spanning tree method is also evaluated with respect to human segmentations. Discrepancy measure is used as best suited to compute the segmentation error between the methods. The evaluation is done using gray value images. It is shown that the segmentation results of these methods have a considerable difference.

Approximative graph pyramid solution of the E-TSP

The traveling salesman problem (TSP) is difficult to solve for input instances with large number of cities. Instead of finding the solution for an input with a large number of cities, the problem is transformed into a simpler form containing smaller number of cities, which is then solved optimally. Graph pyramid solution strategies, using Boruvkas minimum spanning tree step, convert, in a bottom-up processing, a 2D Euclidean TSP problem with a large number of cities into successively smaller problems (graphs) with similar layout and solution, until the number of cities is small enough to seek the optimal solution. Expanding this tour solution in a top-down manner, to the lower levels of the pyramid, leads to an approximate solution. The new model has an adaptive spatial structure and it simulates visual acuity and visual attention. The model solves the TSP problem sequentially, by moving attention from city to city, and the quality of the solutions is similar to the solutions produced by humans. The graph pyramid data structures and processing strategies provide good methods for finding near-optimal solutions for computationally hard problems. Isolating processing used by humans to solve computationally hard problems is of general importance to psychology community and might lead to advances in pattern recognition.