Jonas August

Ligature Instabilities in the Perceptual Organization of Shape

Although the classical Blum skeleton has long been considered unstable, many have attempted to alleviate this defect through pruning. Unfortunately, these methods have an arbitrary basis, and, more importantly, they do not prevent internal structural alterations due to slight changes in an objects boundary. The result is a relative lack of development of skeleton representations for indexing object databases, despite a long history. Here we revisit a subset of the skeleton?called ligature by Blum?to demonstrate how the topological sensitivity of the skeleton can be alleviated. In particular, we show how the deletion of ligature regions leads to stable hierarchical descriptions, illustrating this point with several computational examples. We then relate ligature to a natural growth principle to provide an account of the perceptual parts of shape. Finally, we discuss the duality between the problems of part decomposition and contour fragment grouping.

High-zoom video hallucination by exploiting spatio-temporal regularities

In this paper, we consider the problem of super-resolving a human face video by a very high (/spl times/ 16) zoom factor. Inspired by the literature on hallucination and example-based learning, we formulate this task using a graphical model that encodes, (1) spatio-temporal consistencies, and (2) image formation & degradation processes. A video database of facial expressions is used to learn a domain-specific prior for high-resolution videos. The problem is posed as one of probabilistic inference, in which we aim to find the high-resolution video that satisfies the constraints expressed through the graphical model. Traditional approaches to this problem using video data first estimate the relative motion between frames and then compensate for it, and effectively resulting in multiple measurements of the scene. Our use of time is rather direct, we define data structures that span multiple consecutive frames enriching our feature vectors with a temporal signature. We then exploit these signatures to find consistent solutions over time. In our experiments, an 8/spl times/6 pixel-wide face video, subject to translational jitter and additive noise, gets magnified to a 128/spl times/96 pixel video. Our results show that by exploiting both space and time, drastic improvements can be achieved in both video flicker artifacts and mean-squared-error.

Exploiting inference for approximate parameter learning in discriminative fields: an empirical study

Estimation of parameters of random field models from labeled training data is crucial for their good performance in many image analysis applications. In this paper, we present an approach for approximate maximum likelihood parameter learning in discriminative field models, which is based on approximating true expectations with simple piecewise constant functions constructed using inference techniques. Gradient ascent with these updates exhibits compelling limit cycle behavior which is tied closely to the number of errors made during inference. The performance of various approximations was evaluated with different inference techniques showing that the learned parameters lead to good classification performance so long as the method used for approximating the gradient is consistent with the inference mechanism. The proposed approach is general enough to be used for the training of, e.g., smoothing parameters of conventional Markov Random Fields (MRFs).

The Curve Indicator Random Field: Curve Organization Via Edge Correlation

Can the organization of local edge measurements into curves be directly related to natural image structure? By viewing curve organization as a statistical estimation problem, we suggest that it can. In particular, the classical Gestalt perceptual organization cues of proximity and good continuation—the basis of many current curve organization systems—can be statistically measured in images. As a prior for our estimation approach we introduce the curve indicator random field. In contrast to other techniques that require contour closure or are based on a sparse set of detected edges, the curve indicator random field emphasizes the short-distance, dense nature of organizing curve elements into (possibly) open curves. Its explicit formulation allows the calculation of its properties such as its autocorrelation. On the one hand, the curve indicator random field leads us to introduce the oriented Wiener filter, capturing the blur and noise inherent in the edge measurement process. On the other, it suggests we seek such correlations in natural images. We present the results of some initial edge correlation measurements that not only confirm the presence of Gestalt cues, but also suggest that curvature has a role in curve organization.

Ligature instabilities in the perceptual organization of shape

Although the classical Blum skeleton has long been considered unstable, many have attempted to alleviate this defect through pruning. Unfortunately, these methods have an arbitrary basis, and, more importantly, they do not prevent internal structural alterations due to slight changes in an objects boundary. The result is a relative lack of development of skeleton representations for indexing object databases, despite a long history. Here we revisit a subset of the skeleton-called ligature by Blum-to demonstrate how the topological sensitivity of the skeleton can be eliminated. We relate ligature to a natural growth principle to provide an account of the perceptual parts of shape.

Contour Fragment Grouping and Shared, Simple Occluders

Bounding contours of physical objects are often fragmented by other occluding objects. Long-distance perceptual grouping seeks to join fragments belonging to the same object. Approaches to grouping based on invariants assume objects are in restricted classes, while those based on minimal energy continuations assume a shape for the missing contours and require this shape to drive the grouping process. While these assumptions may be appropriate for certain specific tasks or when contour gaps are small, in general occlusion can give rise to large gaps, and thus long-distance contour fragment grouping is a different type of perceptual organization problem. We propose the long-distance principle that those fragments should be grouped whose fragmentation could have arisen from a shared, simple occluder. The gap skeleton is introduced as a representation of this virtual occluder, and an algorithm for computing it is given. Finally, we show that a view of the virtual occluder as a disk can be interpreted as an equivalence class of curves interpolating the fragment endpoints.

Fragment grouping via the principle of perceptual occlusion

Bounding contours of physical objects are often fragmented by other occluding objects. Long-distance perceptual grouping seeks to join fragments belonging to the same object. Approaches to grouping based on invariants assume objects are in restricted classes, while those based on minimal energy continuations assume a shape for the missing contours and require this shape to drive the grouping process. We propose the more general principle that those fragments should be grouped whose fragmentation could have arisen from a generic occluder. The gap skeleton is introduced as a representation of this virtual occluder, and an algorithm for computing it is given.

Volterra Filtering of Noisy Images of Curves

How should one filter very noisy images of curves? While blurring with a Gaussian reduces noise, it also reduces contour contrast. Both non-homogeneous and anisotropic diffusion smooth images while preserving contours, but these methods assume a single local orientation and therefore they can merge or distort nearby or crossing contours. To avoid these difficulties, we view curve enhancement as a statistical estimation problem in the three-dimensional (x, y, ?)-space of positions and directions, where our prior is a probabilistic model of an ideal edge/line map known as the curve indicator random field (CIRF). Technically, this random field is a superposition of local times of Markov processes that model the individual curves; intuitively, it is an idealized artists sketch, where the value of the field is the amount of ink deposited by the artists pen. After reviewing the CIRF framework and our earlier formulas for the CIRF cumulants, we compute the minimum mean squared error (MMSE) estimate of the CIRF embedded in large amounts of Gaussian white noise. The derivation involves a perturbation expansion in an infinite noise limit, and results in linear, quadratic, and cubic (Volterra) CIRF filters for enhancing images of contours. The self-avoidingness of smooth curves in (x, y, ?) simplified our analysis and the resulting algorithms, which run in O(n log n) time, where n is the size of the input. This suggests that the Gestalt principle of good continuation may not only express the likely smoothness of contours, but it may have a computational basis as well.

A Markov Process Using Curvature for Filtering Curve Images

A Markov process model for contour curvature is introduced via a stochastic differential equation. We analyze the distribution of such curves, and show that its mode is the Euler spiral, a curve minimizing changes in curvature. To probabilistically enhance noisy and low contrast curve images (e.g., edge and line operator responses), we combine this curvature process with the curve indicator random field, which is a prior for ideal curve images. In particular, we provide an expression for a nonlinear, minimum mean square error filter that requires the solution of two elliptic partial differential equations. Initial computations are reported, highlighting how the filter is curvature-selective, even when curvature is absent in the input.

The role of non-overlap in image registration

Here we model the effect of non-overlapping voxels on image registration, and show that a major defect of overlap-only models--their limited capture range--can be alleviated. Theoretically, we introduce a maximum likelihood model that combines histograms of overlapping and non-overlapping voxels into a common joint distribution. The convex problem for the joint distribution is solved via iterative application of replicator equations that converge monotonically. We then focus on rigidly aligning images with unknown translation, where we present a fast FFT-based method for computing joint histograms for all relative translations of an image pair. We then apply this method to standard overlap-only information theoretic registration criteria such as mutual information as well as to our variants that exploit non-overlap. Our experimental results show that global optima correspond to the correct registration generally only when non-overlapping image regions are included.