Alan L. Yuille

Region competition: unifying snakes, region growing, and Bayes/MDL for multiband image segmentation

We present a novel statistical and variational approach to image segmentation based on a new algorithm, named region competition. This algorithm is derived by minimizing a generalized Bayes/minimum description length (MDL) criterion using the variational principle. The algorithm is guaranteed to converge to a local minimum and combines aspects of snakes/balloons and region growing. The classic snakes/balloons and region growing algorithms can be directly derived from our approach. We provide theoretical analysis of region competition including accuracy of boundary location, criteria for initial conditions, and the relationship to edge detection using filters. It is straightforward to generalize the algorithm to multiband segmentation and we demonstrate it on gray level images, color images and texture images. The novel color model allows us to eliminate intensity gradients and shadows, thereby obtaining segmentation based on the albedos of objects. It also helps detect highlight regions.

Feature extraction from faces using deformable templates

We propose a method for detecting and describing features of faces using deformable templates. The feature of interest, an eye for example, is described by a parameterized template. An energy function is defined which links edges, peaks, and valleys in the image intensity to corresponding properties of the template. The template then interacts dynamically with the image by altering its parameter values to minimize the energy function, thereby deforming itself to find the best fit. The final parametr values can be used as descriptors for the feature. We illustrate this method by showing deformable templates detecting eyes and mouths in real images. We demonstrate their ability for tracking features.

The concave-convex procedure

The concave-convex procedure (CCCP) is a way to construct discrete-time iterative dynamical systems that are guaranteed to decrease global optimization and energy functions monotonically. This procedure can be applied to almost any optimization problem, and many existing algorithms can be interpreted in terms of it. In particular, we prove that all expectation-maximization algorithms and classes of Legendre minimization and variational bounding algorithms can be reexpressed in terms of CCCP. We show that many existing neural network and mean-field theory algorithms are also examples of CCCP. The generalized iterative scaling algorithm and Sinkhorns algorithm can also be expressed as CCCP by changing variables. CCCP can be used both as a new way to understand, and prove the convergence of, existing optimization algorithms and as a procedure for generating new algorithms.

Scaling Theorems for Zero Crossings

We characterize some properties of the zero crossings of the Laplacian of signals¿in particular images¿filtered with linear filters, as a function of the scale of the filter (extending recent work by Witkin [16]). We prove that in any dimension the only filter that does not create generic zero crossings as the scale increases is the Gaussian. This result can be generalized to apply to level crossings of any linear differential operator: it applies in particular to ridges and ravines in the image intensity. In the case of the second derivative along the gradient, there is no filter that avoids creation of zero crossings, unless the filtering is performed after the derivative is applied.

Detecting and reading text in natural scenes

This paper gives an algorithm for detecting and reading text in natural images. The algorithm is intended for use by blind and visually impaired subjects walking through city scenes. We first obtain a dataset of city images taken by blind and normally sighted subjects. From this dataset, we manually label and extract the text regions. Next we perform statistical analysis of the text regions to determine which image features are reliable indicators of text and have low entropy (i.e. feature response is similar for all text images). We obtain weak classifiers by using joint probabilities for feature responses on and off text. These weak classifiers are used as input to an AdaBoost machine learning algorithm to train a strong classifier. In practice, we trained a cascade with 4 strong classifiers containing 79 features. An adaptive binarization and extension algorithm is applied to those regions selected by the cascade classifier. Commercial OCR software is used to read the text or reject it as a non-text region. The overall algorithm has a success rate of over 90% (evaluated by complete detection and reading of the text) on the test set and the unread text is typically small and distant from the viewer.

Image Parsing: Unifying Segmentation, Detection, and Recognition

In this paper we present a Bayesian framework for parsing images into their constituent visual patterns. The parsing algorithm optimizes the posterior probability and outputs a scene representation as a “parsing graph”, in a spirit similar to parsing sentences in speech and natural language. The algorithm constructs the parsing graph and re-configures it dynamically using a set of moves, which are mostly reversible Markov chain jumps. This computational framework integrates two popular inference approaches—generative (top-down) methods and discriminative (bottom-up) methods. The former formulates the posterior probability in terms of generative models for images defined by likelihood functions and priors. The latter computes discriminative probabilities based on a sequence (cascade) of bottom-up tests/filters. In our Markov chain algorithm design, the posterior probability, defined by the generative models, is the invariant (target) probability for the Markov chain, and the discriminative probabilities are used to construct proposal probabilities to drive the Markov chain. Intuitively, the bottom-up discriminative probabilities activate top-down generative models. In this paper, we focus on two types of visual patterns—generic visual patterns, such as texture and shading, and object patterns including human faces and text. These types of patterns compete and cooperate to explain the image and so image parsing unifies image segmentation, object detection, and recognition (if we use generic visual patterns only then image parsing will correspond to image segmentation (Tu and Zhu, 2002. IEEE Trans. PAMI, 24(5):657–673). We illustrate our algorithm on natural images of complex city scenes and show examples where image segmentation can be improved by allowing object specific knowledge to disambiguate low-level segmentation cues, and conversely where object detection can be improved by using generic visual patterns to explain away shadows and occlusions.

Data Fusion for Sensory Information Processing Systems

1 Introduction: The Role of Data Fusion in Sensory Systems.- 2 Bayesian Sensory Information Processing.- 3 Information Processing Using Energy Function Minimization.- 4 Weakly vs. Strongly Coupled Data Fusion: A Classification of Fusional Methods.- 5 Data Fusion Applied to Feature Based Stereo Algorithms.- 6 Fusing Binocular and Monocular Depth Cues.- 7 Data Fusion in Shape from Shading Algorithms.- 8 Temporal Aspects of Data Fusion.- 9 Towards a Constraint Based Theory of Sensory Data Fusion.

The bas-relief ambiguity

AbstractWhen an unknown object with Lambertian reflectance is viewed orthographically, there is an implicit ambiguity in determining its 3-d structure: we show that the objects visible surface f(x, y) is indistinguishable from a “generalized bas-relief” transformation of the objects geometry,

Region competition: unifying snakes, region growing, energy/Bayes/MDL for multi-band image segmentation

{\bar f}