Brian Potetz

Efficient Belief Propagation for Vision Using Linear Constraint Nodes

Belief propagation over pairwise connected Markov random fields has become a widely used approach, and has been successfully applied to several important computer vision problems. However, pairwise interactions are often insufficient to capture the full statistics of the problem. Higher-order interactions are sometimes required. Unfortunately, the complexity of belief propagation is exponential in the size of the largest clique. In this paper, we introduce a new technique to compute belief propagation messages in time linear with respect to clique size for a large class of potential functions over real-valued variables. We demonstrate this technique in two applications. First, we perform efficient inference in graphical models where the spatial prior of natural images is captured by 2 times 2 cliques. This approach shows significant improvement over the commonly used pairwise-connected models, and may benefit a variety of applications using belief propagation to infer images or range images. Finally, we apply these techniques to shape-from-shading and demonstrate significant improvement over previous methods, both in quality and in flexibility.

Efficient belief propagation for higher-order cliques using linear constraint nodes

Belief propagation over pairwise-connected Markov random fields has become a widely used approach, and has been successfully applied to several important computer vision problems. However, pairwise interactions are often insufficient to capture the full statistics of the problem. Higher-order interactions are sometimes required. Unfortunately, the complexity of belief propagation is exponential in the size of the largest clique. In this paper, we introduce a new technique to compute belief propagation messages in time linear with respect to clique size for a large class of potential functions over real-valued variables. We discuss how this technique can be generalized to still wider classes of potential functions at varying levels of efficiency. Also, we develop a form of nonparametric belief representation specifically designed to address issues common to networks with higher-order cliques and also to the use of guaranteed-convergent forms of belief propagation. To illustrate these techniques, we perform efficient inference in graphical models where the spatial prior of natural images is captured by 2x2 cliques. This approach shows significant improvement over the commonly used pairwise-connected models, and may benefit a variety of applications using belief propagation to infer images or range images, including stereo, shape-from-shading, image-based rendering, segmentation, and matting.

Statistical correlations between two-dimensional images and three-dimensional structures in natural scenes

In spite of the recent surge in the popularity of statistical approaches to vision, the joint statistics of coregistered range and light-intensity images have gone relatively unexplored. We investigate statistical correlations between images and the surface shapes that produced them. We determine which linear properties of range images can be best predicted from simple computations on intensity information, and we determine those properties of intensity images that best predict range information. We find that significant (up to p = 0.45) and potentially exploitable correlations exist between linear properties of range and intensity images, and we explore the structure of these correlations.

Cooperative and Competitive Interactions Facilitate Stereo Computations in Macaque Primary Visual Cortex

Inferring depth from binocular disparities is a difficult problem for the visual system because local features in the left- and right-eye images must be matched correctly to solve this “stereo correspondence problem.” Cortical architecture and computational studies suggest that lateral interactions among neurons could help resolve local uncertainty about disparity encoded in individual neurons by incorporating contextual constraints. We found that correlated activity among pairs of neurons in primary visual cortex depended both on disparity-tuning relationships and the stimuli displayed within the receptive fields of the neurons. Nearby pairs of neurons with distinct disparity tuning exhibited a decrease in spike correlation at competing disparities soon after response onset. Distant neuronal pairs of similar disparity tuning exhibited an increase in spike correlation at mutually preferred disparities. The observed correlated activity and response dynamics suggests that local competitive and distant cooperative interactions improve disparity tuning of individual neurons over time. Such interactions could represent a neural substrate for the principal constraints underlying cooperative stereo algorithms.

Relative luminance and binocular disparity preferences are correlated in macaque primary visual cortex, matching natural scene statistics

Humans excel at inferring information about 3D scenes from their 2D images projected on the retinas, using a wide range of depth cues. One example of such inference is the tendency for observers to perceive lighter image regions as closer. This psychophysical behavior could have an ecological basis because nearer regions tend to be lighter in natural 3D scenes. Here, we show that an analogous association exists between the relative luminance and binocular disparity preferences of neurons in macaque primary visual cortex. The joint coding of relative luminance and binocular disparity at the neuronal population level may be an integral part of the neural mechanisms for perceptual inference of depth from images.

The role of spiking nonlinearity in contrast gain control and information transmission

Threshold and saturation are two nonlinear features common to almost all spiking neurons. How these nonlinearities affect the performance gain of the transfer function and coding properties of the neurons has attracted much attention. Here, we deduce basic analytical relationships among these nonlinearities (threshold and saturation), performance gain and information transmission in neurons. We found that performance gain and information transmission can be maximized by input signals with optimal variance. The threshold and saturation inside the model determines the gain tuning property and maximum coding capacity. This framework provides an understanding of some basic design principles underlying information processing systems that can be adjusted to match the statistics of signals in the environment. This study also isolates the exact contributions of the nonlinearities on the contrast adaptation phenomena observed in real visual neurons.

Sample skewness as a statistical measurement of neuronal tuning sharpness

We propose using the statistical measurement of the sample skewness of the distribution of mean firing rates of a tuning curve to quantify sharpness of tuning. For some features, like binocular disparity, tuning curves are best described by relatively complex and sometimes diverse functions, making it difficult to quantify sharpness with a single function and parameter. Skewness provides a robust nonparametric measure of tuning curve sharpness that is invariant with respect to the mean and variance of the tuning curve and is straightforward to apply to a wide range of tuning, including simple orientation tuning curves and complex object tuning curves that often cannot even be described parametrically. Because skewness does not depend on a specific model or function of tuning, it is especially appealing to cases of sharpening where recurrent interactions among neurons produce sharper tuning curves that deviate in a complex manner from the feedforward function of tuning. Since tuning curves for all neurons are not typically well described by a single parametric function, this model independence additionally allows skewness to be applied to all recorded neurons, maximizing the statistical power of a set of data. We also compare skewness with other nonparametric measures of tuning curve sharpness and selectivity. Compared to these other nonparametric measures tested, skewness is best used for capturing the sharpness of multimodal tuning curves defined by narrow peaks (maximum) and broad valleys (minima). Finally, we provide a more formal definition of sharpness using a shape-based information gain measure and derive and show that skewness is correlated with this definition.

Spatial anisotropies and temporal fluctuations in extracellular matrix network texture during early embryogenesis.

Early stages of vertebrate embryogenesis are characterized by a remarkable series of shape changes. The resulting morphological complexity is driven by molecular, cellular, and tissue-scale biophysical alterations. Operating at the cellular level, extracellular matrix (ECM) networks facilitate cell motility. At the tissue level, ECM networks provide material properties required to accommodate the large-scale deformations and forces that shape amniote embryos. In other words, the primordial biomaterial from which reptilian, avian, and mammalian embryos are molded is a dynamic composite comprised of cells and ECM. Despite its central importance during early morphogenesis we know little about the intrinsic micrometer-scale surface properties of primordial ECM networks. Here we computed, using avian embryos, five textural properties of fluorescently tagged ECM networks — (a) inertia, (b) correlation, (c) uniformity, (d) homogeneity, and (e) entropy. We analyzed fibronectin and fibrillin-2 as examples of fibrous ECM constituents. Our quantitative data demonstrated differences in the surface texture between the fibronectin and fibrillin-2 network in Day 1 (gastrulating) embryos, with the fibronectin network being relatively coarse compared to the fibrillin-2 network. Stage-specific regional anisotropy in fibronectin texture was also discovered. Relatively smooth fibronectin texture was exhibited in medial regions adjoining the primitive streak (PS) compared with the fibronectin network investing the lateral plate mesoderm (LPM), at embryonic stage 5. However, the texture differences had changed by embryonic stage 6, with the LPM fibronectin network exhibiting a relatively smooth texture compared with the medial PS-oriented network. Our data identify, and partially characterize, stage-specific regional anisotropy of fibronectin texture within tissues of a warm-blooded embryo. The data suggest that changes in ECM textural properties reflect orderly time-dependent rearrangements of a primordial biomaterial. We conclude that the ECM microenvironment changes markedly in time and space during the most important period of amniote morphogenesis—as determined by fluctuating textural properties.

Bayesian Classifiers for Chemical Toxicity Prediction

A major concern across the globe is the growing number of new chemicals that are brought to use on a regular basis without having any knowledge about their toxic behavior. The challenge here is that the growth in the number of chemicals is fast, and the traditional standards for toxicity testing involve a slow and expensive process of in vivo animal testing. Hence, a number of attempts are being made to find alternate methods of toxicity testing. In this paper we explore Bayesian classifiers and show that if we approximate posterior in the Bayesian classifier with specially crafted basis functions, we can improve upon the performance. We have tested our methods using data sets from the Environmental Protection Agency (EPA). Our experimental study demonstrated the utility of the advanced Bayesian classification approach.

Object Categorization: Neural Encoding of Scene Statistics for Surface and Object Inference

Features associated with an object or its surfaces in natural scenes tend to vary coherently in space and time. In psychological literature, these coherent covariations have been described as important for neural systems to acquire models of objects and object categories. From a statistical inference perspective, such coherent covariation can provide a mechanism to learn statistical priors in natural scenes that are useful for probabilistic inference. In this article, we present some neurophysiological experimental observations in the early visual cortex that provide insights into how correlation structures in visual scenes are being encoded by neuronal tuning and connections among neurons. The key insight is that correlated structures in visual scenes result in correlated neuronal activities, which shapes the tuning properties of individual neurons and the connections between them, embedding Gestalt-related computational constraints or priors for surface inference. Extending these concepts to the inferotemporal cortex suggests a representational framework that is distinct from the traditional feed-forward hierarchy of invariant object representation and recognition. In this framework, lateral connections among view-based neurons, learned from the temporal association of the object views observed over time, can form a linked graph structure with local dependency, akin to a dense aspect graph in computer vision. This web-like graph allows view-invariant object representation to be created using sparse feed-forward connections, while maintaining the explicit representation of the different views. Thus, it can serve as an effective prior model for generating predictions of future incoming views to facilitate object inference.