Tai Sing Lee

Hierarchical Bayesian Inference in the Visual Cortex

Traditional views of visual processing suggest that early visual neurons in areas V1 and V2 are static spatiotemporal filters that extract local features from a visual scene. The extracted information is then channeled through a feedforward chain of modules in successively higher visual areas for further analysis. Recent electrophysiological recordings from early visual neurons in awake behaving monkeys reveal that there are many levels of complexity in the information processing of the early visual cortex, as seen in the long-latency responses of its neurons. These new findings suggest that activity in the early visual cortex is tightly coupled and highly interactive with the rest of the visual system. They lead us to propose a new theoretical setting based on the mathematical framework of hierarchical Bayesian inference for reasoning about the visual system. In this framework, the recurrent feedforward/feedback loops in the cortex serve to integrate top-down contextual priors and bottom-up observations so as to implement concurrent probabilistic inference along the visual hierarchy. We suggest that the algorithms of particle filtering and Bayesian-belief propagation might model these interactive cortical computations. We review some recent neurophysiological evidences that support the plausibility of these ideas.

Region competition: unifying snakes, region growing, energy/Bayes/MDL for multi-band image segmentation

We present a novel statistical and variational approach to image segmentation based on a new algorithm named region competition. This algorithm is derived by minimizing a generalized Bayes/MDL (Minimum Description Length) criterion using the variational principle. We show that existing techniques in early vision such as, snake/balloon models, region growing, and Bayes/MDL are addressing different aspects of the same problem and they can be unified within a common statistical framework which combines their advantages. We analyze how to optimize the precision of the resulting boundary location by studying the statistical properties of the region competition algorithm and discuss what are good initial conditions for the algorithm. Our method is generalized to color and texture segmentation and is demonstrated on grey level images, color images and texture images.<<ETX>>

Comparison of recordings from microelectrode arrays and single electrodes in the visual cortex

Advances in microelectrode neural recording systems have made it possible to record extracellular activity from a large number of neurons simultaneously. A substantial body of work is associated with traditional single-electrode extracellular recording, and the robustness of the recording method has

A unified model of spatial and object attention based on inter-cortical biased competition

Abstract We present a physiologically constrained neural dynamical model of the visual system for the organization of attention and its mediation of object recognition and visual search. In this model, spatial and feature attention are mediated by a single neural mechanism involving the interaction of the ventral and the dorsal streams with the early visual cortex. The model consists of three representative modules which encode object classes, spatial locations, and elementary features, respectively. These modules are coupled together in a neural dynamical system in the framework of biased competition. The system can be made to operate in either a spatial or an object attention mode by introducing a top-down bias to either the dorsal or the ventral stream modules. In this system, translation invariant object recognition and object spatial localization arise from the interaction among the modules, with the early visual areas playing a key role in mediating such interaction.

Local field potentials indicate network state and account for neuronal response variability

Multineuronal recordings have revealed that neurons in primary visual cortex (V1) exhibit coordinated fluctuations of spiking activity in the absence and in the presence of visual stimulation. From the perspective of understanding a single cell’s spiking activity relative to a behavior or stimulus, these network fluctuations are typically considered to be noise. We show that these events are highly correlated with another commonly recorded signal, the local field potential (LFP), and are also likely related to global network state phenomena which have been observed in a number of neural systems. Moreover, we show that attributing a component of cell firing to these network fluctuations via explicit modeling of the LFP improves the recovery of cell properties. This suggests that the impact of network fluctuations may be estimated using the LFP, and that a portion of this network activity is unrelated to the stimulus and instead reflects ongoing cortical activity. Thus, the LFP acts as an easily accessible bridge between the network state and the spiking activity.

Computations in the early visual cortex

This paper reviews some of the recent neurophysiological studies that explore the variety of visual computations in the early visual cortex in relation to geometric inference, i.e. the inference of contours, surfaces and shapes. It attempts to draw connections between ideas from computational vision and findings from awake primate electrophysiology. In the classical feed-forward, modular view of visual processing, the early visual areas (LGN, V1 and V2) are modules that serve to extract local features, while higher extrastriate areas are responsible for shape inference and invariant object recognition. However, recent findings in primate early visual systems reveal that the computations in the early visual cortex are rather complex and dynamic, as well as interactive and plastic, subject to influence from global context, higher order perceptual inference, task requirement and behavioral experience. The evidence argues that the early visual cortex does not merely participate in the first stage of visual processing, but is involved in many levels of visual computation.

Efficient belief propagation for higher-order cliques using linear constraint nodes

Belief propagation over pairwise-connected Markov random fields has become a widely used approach, and has been successfully applied to several important computer vision problems. However, pairwise interactions are often insufficient to capture the full statistics of the problem. Higher-order interactions are sometimes required. Unfortunately, the complexity of belief propagation is exponential in the size of the largest clique. In this paper, we introduce a new technique to compute belief propagation messages in time linear with respect to clique size for a large class of potential functions over real-valued variables. We discuss how this technique can be generalized to still wider classes of potential functions at varying levels of efficiency. Also, we develop a form of nonparametric belief representation specifically designed to address issues common to networks with higher-order cliques and also to the use of guaranteed-convergent forms of belief propagation. To illustrate these techniques, we perform efficient inference in graphical models where the spatial prior of natural images is captured by 2x2 cliques. This approach shows significant improvement over the commonly used pairwise-connected models, and may benefit a variety of applications using belief propagation to infer images or range images, including stereo, shape-from-shading, image-based rendering, segmentation, and matting.

The role of early visual cortex in visual integration: a neural model of recurrent interaction

This paper presents a model on the potential functional roles of the early visual cortex in the primate visual system. Our hypothesis is that early visual areas, such as V1, are important for continual interaction among various higher order visual areas during visual processing. The interaction is mediated by recurrent connections between higher order visual areas and V1, manifested in the long‐latency context‐sensitive activities often observed in neurophysiological experiments, and is responsible for the re‐integration of information analysed by the higher visual areas. Specifically, we considered the case of integrating ‘what’ and ‘where’ information from the ventral and dorsal streams. We found that such a cortical architecture provides simple solutions and fresh insights into the problems of attentional routing and visual search. The computational viability of this architecture was tested by simulating a large‐scale neural dynamical network.

Statistical correlations between two-dimensional images and three-dimensional structures in natural scenes

In spite of the recent surge in the popularity of statistical approaches to vision, the joint statistics of coregistered range and light-intensity images have gone relatively unexplored. We investigate statistical correlations between images and the surface shapes that produced them. We determine which linear properties of range images can be best predicted from simple computations on intensity information, and we determine those properties of intensity images that best predict range information. We find that significant (up to p = 0.45) and potentially exploitable correlations exist between linear properties of range and intensity images, and we explore the structure of these correlations.

Texture Segmentation by Minimizing Vector-Valued Energy Functionals: The Coupled-Membrane Model

This paper presents a computational model that segments images based on the textural properties of object surfaces. The proposed Coupled-Membrane model applies the weak membrane approach to an image WI(σ,θ, x, y), derived from the power responses of a family of selfsimilar quadrature Gabor wavelets. While segmentation breaks are allowed in x and y only, coupling is introduced to in all 4 dimensions. The resulting spatial and spectral diffusion prevents minor variations in local textures from producing segmentation boundaries. Experiments showed that the model is adequate in segmenting a class of synthetic and natural texture images.