Wilson S. Geisler

Multichannel texture analysis using localized spatial filters

A computational approach for analyzing visible textures is described. Textures are modeled as irradiance patterns containing a limited range of spatial frequencies, where mutually distinct textures differ significantly in their dominant characterizing frequencies. By encoding images into multiple narrow spatial frequency and orientation channels, the slowly varying channel envelopes (amplitude and phase) are used to segregate textural regions of different spatial frequency, orientation, or phase characteristics. Thus, an interpretation of image texture as a region code, or carrier of region information, is emphasized. The channel filters used, known as the two-dimensional Gabor functions, are useful for these purposes in several senses: they have tunable orientation and radial frequency bandwidths and tunable center frequencies, and they optimally achieve joint resolution in space and in spatial frequency. By comparing the channel amplitude responses, one can detect boundaries between textures. Locating large variations in the channel phase responses allows discontinuities in the texture phase to be detected. Examples are given of both types of texture processing using a variety of real and synthetic textures. >

Image quality assessment based on a degradation model

We model a degraded image as an original image that has been subject to linear frequency distortion and additive noise injection. Since the psychovisual effects of frequency distortion and noise injection are independent, we decouple these two sources of degradation and measure their effect on the human visual system. We develop a distortion measure (DM) of the effect of frequency distortion, and a noise quality measure (NQM) of the effect of additive noise. The NQM, which is based on Pelis (1990) contrast pyramid, takes into account the following: 1) variation in contrast sensitivity with distance, image dimensions, and spatial frequency; 2) variation in the local luminance mean; 3) contrast interaction between spatial frequencies; 4) contrast masking effects. For additive noise, we demonstrate that the nonlinear NQM is a better measure of visual quality than peak signal-to noise ratio (PSNR) and linear quality measures. We compute the DM in three steps. First, we find the frequency distortion in the degraded image. Second, we compute the deviation of this frequency distortion from an allpass response of unity gain (no distortion). Finally, we weight the deviation by a model of the frequency response of the human visual system and integrate over the visible frequencies. We demonstrate how to decouple distortion and additive noise degradation in a practical image restoration system.

Edge co-occurrence in natural images predicts contour grouping performance.

The human brain manages to correctly interpret almost every visual image it receives from the environment. Underlying this ability are contour grouping mechanisms that appropriately link local edge elements into global contours. Although a general view of how the brain achieves effective contour grouping has emerged, there have been a number of different specific proposals and few successes at quantitatively predicting performance. These previous proposals have been developed largely by intuition and computational trial and error. A more principled approach is to begin with an examination of the statistical properties of contours that exist in natural images, because it is these statistics that drove the evolution of the grouping mechanisms. Here we report measurements of both absolute and Bayesian edge co-occurrence statistics in natural images, as well as human performance for detecting natural-shaped contours in complex backgrounds. We find that contour detection performance is quantitatively predicted by a local grouping rule derived directly from the co-occurrence statistics, in combination with a very simple integration rule (a transitivity rule) that links the locally grouped contour elements into longer contours.

Optimal eye movement strategies in visual search

To perform visual search, humans, like many mammals, encode a large field of view with retinas having variable spatial resolution, and then use high-speed eye movements to direct the highest-resolution region, the fovea, towards potential target locations. Good search performance is essential for survival, and hence mammals may have evolved efficient strategies for selecting fixation locations. Here we address two questions: what are the optimal eye movement strategies for a foveated visual system faced with the problem of finding a target in a cluttered environment, and do humans employ optimal eye movement strategies during a search? We derive the ideal bayesian observer for search tasks in which a target is embedded at an unknown location within a random background that has the spectral characteristics of natural scenes. Our ideal searcher uses precise knowledge about the statistics of the scenes in which the target is embedded, and about its own visual system, to make eye movements that gain the most information about target location. We find that humans achieve nearly optimal search performance, even though humans integrate information poorly across fixations. Analysis of the ideal searcher reveals that there is little benefit from perfect integration across fixations—much more important is efficient processing of information on each fixation. Apparently, evolution has exploited this fact to achieve efficient eye movement strategies with minimal neural resources devoted to memory.

Spikes versus BOLD: what does neuroimaging tell us about neuronal activity?

By demonstrating that fMRI responses in human MT+ increase linearly with motion coherence and comparing these responses with slopes of single-neuron firing rates in monkey MT, a new paper provides the best evidence so far that fMRI responses are proportional to firing rates.

Independence of luminance and contrast in natural scenes and in the early visual system.

The early visual system is endowed with adaptive mechanisms that rapidly adjust gain and integration time based on the local luminance (mean intensity) and contrast (standard deviation of intensity relative to the mean). Here we show that these mechanisms are matched to the statistics of the environment. First, we measured the joint distribution of luminance and contrast in patches selected from natural images and found that luminance and contrast were statistically independent of each other. This independence did not hold for artificial images with matched spectral characteristics. Second, we characterized the effects of the adaptive mechanisms in lateral geniculate nucleus (LGN), the direct recipient of retinal outputs. We found that luminance gain control had the same effect at all contrasts and that contrast gain control had the same effect at all mean luminances. Thus, the adaptive mechanisms for luminance and contrast operate independently, reflecting the very independence encountered in natural images.

Motion streaks provide a spatial code for motion direction

Although many neurons in the primary visual cortex (V1) of primates are direction selective, they provide ambiguous information about the direction of motion of a stimulus,. There is evidence that one of the ways in which the visual system resolves this ambiguity is by computing, from the responses of V1 neurons, velocity components in two or more spatial orientations and then combining these velocity components,,,,,,,. Here I consider another potential neural mechanism for determining motion direction. When a localized image feature moves fast enough, it should become smeared in space owing to temporal integration in the visual system, creating a spatial signal—a ‘motion streak’—oriented in the direction of the motion. The orientation masking and adaptation experiments reported here show that these spatial signals for motion direction exist in the human visual system for feature speeds above about 1 feature width per 100 ms. Computer simulations show that this psychophysical finding is consistent with the known response properties of V1 neurons, and that these spatial signals, when appropriately processed, are sufficient to determine motion direction in natural images.

Visual cortex neurons in monkeys and cats: detection, discrimination, and identification.

A descriptive function method was used to measure the detection, discrimination, and identification performance of a large population of single neurons recorded from within the primary visual cortex of the monkey and the cat, along six stimulus dimensions: contrast, spatial position, orientation, spatial frequency, temporal frequency, and direction of motion. First, the responses of single neurons were measured along each stimulus dimension, using analysis intervals comparable to a normal fixation interval (200 ms). Second, the measured responses of each neuron were fitted with simple descriptive functions, containing a few free parameters, for each stimulus dimension. These functions were found to account for approximately 90% of the variance in the measured response means and response standard deviations. (A detailed analysis of the relationship between the mean and the variance showed that the variance is proportional to the mean.) Third, the parameters of the best-fitting descriptive functions were utilized in conjunction with Bayesian (optimal) decision theory to determine the detection, discrimination, and identification performance for each neuron, along each stimulus dimension. For some of the cells in monkey, discrimination performance was comparable to behavioral performance; for most of the cells in cat, discrimination performance was better than behavioral performance. The behavioral contrast and spatial-frequency discrimination functions were similar in shape to the envelope of the most sensitive cells; they were also similar to the discrimination functions obtained by optimal pooling of the entire population of cells. The statistics which summarize the parameters of the descriptive functions were used to estimate the response of the visual cortex as a whole to a complex natural image. The analysis suggests that individual cortical neurons can reliably signal precise information about the location, size, and orientation of local image features.

Cortical neurons: Isolation of contrast gain control

The selectivity of cortical neurons remains invariant with contrast, even though the contrast-response function saturates. Both the invariance and the saturation might be due to a contrast-gain control mechanism. To test this hypothesis, a drifting grafting was used to measure the contrast-response function, while a counterphase grating was simultaneously presented at the null position of the receptive field (where it evokes no response at any contrast). When the contrast of the counterphase grating increased, the contrast-response function shifted primarily to the right. This result is consistent with the hypothesis that there is a fast-acting gain-control mechanism which effectively scales the input contrast by the average local contrast.

Illusions, perception and Bayes.

A new model shows that a range of visual illusions in humans can be explained as rational inferences about the odds that a motion stimulus on the retina results from a particular real-world source.