Odelia Schwartz

Natural signal statistics and sensory gain control.

We describe a form of nonlinear decomposition that is well-suited for efficient encoding of natural signals. Signals are initially decomposed using a bank of linear filters. Each filter response is then rectified and divided by a weighted sum of rectified responses of neighboring filters. We show that this decomposition, with parameters optimized for the statistics of a generic ensemble of natural images or sounds, provides a good characterization of the nonlinear response properties of typical neurons in primary visual cortex or auditory nerve, respectively. These results suggest that nonlinear response properties of sensory neurons are not an accident of biological implementation, but have an important functional role.

Spatiotemporal Elements of Macaque V1 Receptive Fields

Neurons in primary visual cortex (V1) are commonly classified as simple or complex based upon their sensitivity to the sign of stimulus contrast. The responses of both cell types can be described by a general model in which the outputs of a set of linear filters are nonlinearly combined. We estimated the model for a population of V1 neurons by analyzing the mean and covariance of the spatiotemporal distribution of random bar stimuli that were associated with spikes. This analysis reveals an unsuspected richness of neuronal computation within V1. Specifically, simple and complex cell responses are best described using more linear filters than the one or two found in standard models. Many filters revealed by the model contribute suppressive signals that appear to have a predominantly divisive influence on neuronal firing. Suppressive signals are especially potent in direction-selective cells, where they reduce responses to stimuli moving in the nonpreferred direction.

Spike-triggered neural characterization

Response properties of sensory neurons are commonly described using receptive fields. This description may be formalized in a model that operates with a small set of linear filters whose outputs are nonlinearly combined to determine the instantaneous firing rate. Spike-triggered average and covariance analyses can be used to estimate the filters and nonlinear combination rule from extracellular experimental data. We describe this methodology, demonstrating it with simulated model neuron examples that emphasize practical issues that arise in experimental situations.

Space and time in visual context

No sensory stimulus is an island unto itself; rather, it can only properly be interpreted in light of the stimuli that surround it in space and time. This can result in entertaining illusions and puzzling results in psychological and neurophysiological experiments. We concentrate on perhaps the best studied test case, namely orientation or tilt, which gives rise to the notorious tilt illusion and the adaptation tilt after-effect. We review the empirical literature and discuss the computational and statistical ideas that are battling to explain these conundrums, and thereby gain favour as more general accounts of cortical processing.

Perceptual organization in the tilt illusion

The tilt illusion is a paradigmatic example of contextual influences on perception. We analyze it in terms of a neural population model for the perceptual organization of visual orientation. In turn, this is based on a well-found treatment of natural scene statistics, known as the Gaussian Scale Mixture model. This model is closely related to divisive gain control in neural processing and has been extensively applied in the image processing and statistical learning communities; however, its implications for contextual effects in biological vision have not been studied. In our model, oriented neural units associated with surround tilt stimuli participate in divisively normalizing the activities of the units representing a center stimulus, thereby changing their tuning curves. We show that through standard population decoding, these changes lead to the forms of repulsion and attraction observed in the tilt illusion. The issues in our model readily generalize to other visual attributes and contextual phenomena, and should lead to more rigorous treatments of contextual effects based on natural scene statistics.

Spike-triggered characterization of excitatory and suppressive stimulus dimensions in monkey V1

Neurons in primary visual cortex are commonly characterized using linear models, or simple extensions of linear models. Specifically, V1 simple cell responses are often characterized using a rectified linear receptive field, and complex cell responses are often described as the sum of squared responses of two linear subunits. We examined this class of model directly by applying spike-triggered covariance analysis to responses of monkey V1 neurons under binary white noise stimulation. The analysis extracts a low-dimensional subspace of the full stimulus space that is primarily responsible for generation of the neural response, including both excitatory and suppressive components. We found no fewer than two excitatory dimensions in simple cells, and as many as seven dimensions in complex cells. For all cells, we also found suppressive dimensions that were at least equal in number to the excitatory dimensions. These results suggest that extensions to standard models are required to fully describe the response properties of cells in V1.

Cortical Surround Interactions and Perceptual Salience via Natural Scene Statistics

Spatial context in images induces perceptual phenomena associated with salience and modulates the responses of neurons in primary visual cortex (V1). However, the computational and ecological principles underlying contextual effects are incompletely understood. We introduce a model of natural images that includes grouping and segmentation of neighboring features based on their joint statistics, and we interpret the firing rates of V1 neurons as performing optimal recognition in this model. We show that this leads to a substantial generalization of divisive normalization, a computation that is ubiquitous in many neural areas and systems. A main novelty in our model is that the influence of the context on a target stimulus is determined by their degree of statistical dependence. We optimized the parameters of the model on natural image patches, and then simulated neural and perceptual responses on stimuli used in classical experiments. The model reproduces some rich and complex response patterns observed in V1, such as the contrast dependence, orientation tuning and spatial asymmetry of surround suppression, while also allowing for surround facilitation under conditions of weak stimulation. It also mimics the perceptual salience produced by simple displays, and leads to readily testable predictions. Our results provide a principled account of orientation-based contextual modulation in early vision and its sensitivity to the homogeneity and spatial arrangement of inputs, and lends statistical support to the theory that V1 computes visual salience.

Soft Mixer Assignment in a Hierarchical Generative Model of Natural Scene Statistics

Gaussian scale mixture models offer a top-down description of signal generation that captures key bottom-up statistical characteristics of filter responses to images. However, the pattern of dependence among the filters for this class of models is prespecified. We propose a novel extension to the gaussian scale mixturemodel that learns the pattern of dependence from observed inputs and thereby induces a hierarchical representation of these inputs. Specifically, we propose that inputs are generated by gaussian variables (modeling local filter structure), multiplied by a mixer variable that is assigned probabilistically to each input from a set of possible mixers. We demonstrate inference of both components of the generative model, for synthesized data and for different classes of natural images, such as a generic ensemble and faces. For natural images, the mixer variable assignments show invariances resembling those of complex cells in visual cortex; the statistics of the gaussian components of the model are in accord with the outputs of divisive normalization models. We also show how our model helps interrelate a wide range of models of image statistics and cortical processing.

Visuomotor Characterization of Eye Movements in a Drawing Task

Understanding visuomotor coordination requires the study of tasks that engage mechanisms for the integration of visual and motor information; in this paper we choose a paradigmatic yet little studied example of such a task, namely realistic drawing. On the one hand, our data indicate that the motor task has little influence on which regions of the image are overall most likely to be fixated: salient features are fixated most often. Viceversa, the effect of motor constraints is revealed in the temporal aspect of the scanpaths: (1) subjects direct their gaze to an object mostly when they are acting upon (drawing) it; and (2) in support of graphically continuous hand movements, scanpaths resemble edge-following patterns along image contours. For a better understanding of such properties, a computational model is proposed in the form of a novel kind of Dynamic Bayesian Network, and simulation results are compared with human eye-hand data.

Flexible Gating of Contextual Influences in Natural Vision

Identical sensory inputs can be perceived as markedly different when embedded in distinct contexts. Neural responses to simple stimuli are also modulated by context, but the contribution of this modulation to the processing of natural sensory input is unclear. We measured surround suppression, a quintessential contextual influence, in macaque primary visual cortex with natural images. We found that suppression strength varied substantially for different images. This variability was not well explained by existing descriptions of surround suppression, but it was predicted by Bayesian inference about statistical dependencies in images. In this framework, surround suppression was flexible: it was recruited when the image was inferred to contain redundancies and substantially reduced in strength otherwise. Thus, our results reveal a gating of a basic, widespread cortical computation by inference about the statistics of natural input.