George Sperling

Elaborated Reichardt detectors

The elaborated Reichardt detector (ERD) proposed by van Santen and Sperling [J. Opt. Soc. Am. A 1, 451 (1984)], based on Reichardts motion detector [Z. Naturforsch. Teil B 12, 447 (1957)], is an opponent system of two mirror-image subunits. Each subunit receives inputs from two spatiotemporal filters (receptive fields), multiplies the filter outputs, and temporally integrates the product. Subunit outputs are algebraically subtracted to yield ERD output. ERDs can correctly indicate direction of motion of drifting sine waves of any spatial and temporal frequency. Here we prove that with a careful choice of either temporal or spatial filters, the subunits can themselves become quite similar or equivalent to the whole ERD; with suitably chosen filters, the ERD is equivalent to an elaborated version of a motion detector proposed by Watson and Ahumada [NASA Tech. Memo. 84352 (1983)]; and for every choice of filters, the ERD is fully equivalent to the detector proposed by Adelson and Bergen [J. Opt. Soc. Am. A 2, 284-299 (1985)]. Some equivalences between the motion detection (in x, t) by ERDs and spatial pattern detection (in x, y) are demonstrated. The responses of the ERD and its variants to drifting sinusoidal gratings, to other sinusoidally modulated stimuli (on-off gratings, counterphase flicker), and to combinations of sinusoids are derived and compared with data. ERD responses to two-frame motion displays are derived, and several new experimental predictions are tested experimentally.(ABSTRACT TRUNCATED AT 250 WORDS)

A Model for Visual Memory Tasks

A model for visual recall tasks was presented in terms of visual information storage (VIS), scanning, rehearsal, and auditory information storage (AIS). It was shown first that brief visual stimuli are stored in VIS in a form similar to the sensory input. These visual “images” contain considerably more information than is transmitted later. They can be sampled by scanning for items at high rates of about 10 msec per letter. Recall is based on a verbal receding of the stimulus (rehearsal), which is remembered in AIS. The items retained in AIS are usually rehearsed again to prevent them from decaying. The human limits in immediate-memory (reproduction) tasks are inherent in the AIS-Rehearsal loop. The main implication of the model for human factors is the importance of the auditory coding in visual tasks.

Drift-balanced random stimuli: a general basis for studying non-Fourier motion perception

To some degree, all current models of visual motion-perception mechanisms depend on the power of the visual signal in various spatiotemporal-frequency bands. Here we show how to construct counterexamples: visual stimuli that are consistently perceived as obviously moving in a fixed direction yet for which Fourier-domain power analysis yields no systematic motion components in any given direction. We provide a general theoretical framework for investigating non-Fourier motion-perception mechanisms; central are the concepts of drift-balanced and microbalanced random stimuli. A random stimulus S is drift balanced if its expected power in the frequency domain is symmetric with respect to temporal frequency, that is, if the expected power in S of every drifting sinusoidal component is equal to the expected power of the sinusoid of the same spatial frequency, drifting at the same rate in the opposite direction. Additionally, S is microbalanced if the result WS of windowing S by any space-time-separable function W is drift balanced. We prove that (i) any space-time-separable random (or nonrandom) stimulus is microbalanced; (ii) any linear combination of pairwise independent microbalanced (respectively, drift-balanced) random stimuli is microbalanced and drift balanced if the expectation of each component is uniformly zero; (iii) the convolution of independent microbalanced and drift-balanced random stimuli is microbalanced and drift balanced; (iv) the product of independent microbalanced random stimuli is microbalanced; and (v) the expected response of any Reichardt detector to any microbalanced random stimulus is zero at every instant in time. Examples are provided of classes of microbalanced random stimuli that display consistent and compelling motion in one direction. All the results and examples from the domain of motion perception are transposable to the space-domain problem of detecting orientation in a texture pattern.

Successive approximations to a model for short term memory

Abstract Experimental data are considered from a simple task in which an observer looks at letters and then writes them down. Three models are proposed. Model 1 consists of only two components: a visual memory for the letters and a motor translation component to enable copying a visual memory onto paper. Model 1 is inadequate because the visual image is shown not to persist until the time of reproduction. Model 2 corrects this deficiency by incorporating the possibility of subvocal rehearsal of the stimulus letters and an auditory memory for the rehearsal. However, Model 2 cannot account for performance with extremely short duration images because of the limit on the maximum rehearsal rate. The critical improvement in Model 3 is a more detailed specification of scanning, recognition and rehearsal, including a form of memory which is inherent in the process of recognition itself. Model 3 accounts for these data and incidently gives rise to some interesting inferences about the nature of consciousness.

Temporal covariance model of human motion perception

We propose a model of direction-sensitive units in human vision. It is a modified and elaborated version of a model by Reichardt [Z. Naturforsch . Teil B 12, 447 (1957)]. The model is applied to threshold experiments in which subjects view adjacent vertical bars with independently (typically sinusoidally), temporally modulated luminances. The subject must report whether the patterns moved to the left or to the right. According to the model, a basic motion-detecting unit consists of two subunits tuned to opposite directions. Each performs a spatial and temporal linear filtering of its input; outputs of the filters are multiplied, and the multiplied output is integrated (for a time that is long relative to the modulation period). The models output consists of the difference between the subunit outputs. Direction of movement is indicated by the sign of the model output. Mathematical analysis of the model yielded several predictions that were confirmed experimentally. Specifically, we found that (1) performance with complex patterns can be predicted by spatiotemporal Fourier analysis that results in the segregation and linear addition in the output for different temporal frequencies; (2) under special conditions, performance depends on the product of adjacent bar amplitudes, offering strong support for the multiplication principle; (3) performance is unaffected by addition of stationary patterns; and (4) addition of homogeneous flicker normally produces no effect but under special conditions reverses perceived direction. These and other results confirm our model and reject several other models, including Reichardt s original model.

The functional architecture of human visual motion perception.

UNLABELLED A powerful paradigm (the pedestal-plus-test display) is combined with several subsidiary paradigms (interocular presentation, stimulus superpositions with varying phases, and attentional manipulations) to determine the functional architecture of visual motion perception: i.e. the nature of the various mechanisms of motion perception and their relations to each other. Three systems are isolated: a first-order system that uses a primitive motion energy computation to extract motion from moving luminance modulations; a second-order system that uses motion energy to extract motion from moving texture-contrast modulations; and a third-order system that tracks features. Pedestal displays exclude feature-tracking and thereby yield pure measures of the first- and second-order systems which are found to be exclusively monocular. Interocular displays exclude the first- and second-order systems and thereby to yield pure measures of feature-tracking. RESULTS both first- and second-order systems are fast (with temporal frequency cutoff at 12 Hz) and sensitive. Feature tracking operates interocularly almost as well as monocularly. It is slower (cutoff frequency is 3 Hz) and it requires much more stimulus contrast than the first- and second-order systems. Feature tracking is both bottom-up (it computes motion from luminance modulation, texture-contrast modulation, depth modulation, motion modulation, flicker modulation, and from other types of stimuli) and top-down--e.g. attentional instructions can determine the direction of perceived motion.

Model for visual luminance discrimination and flicker detection.

A model for vision is proposed. Its basic units are RC stages whose time constants—in three instances—are parametrically controlled. The requirements of compressing the dynamic range of the input and of fitting luminance pulse-detection data suffice to determine the arrangement and parameters of the components. This model accurately predicts the psychophysical results of flicker detection (DeLange characteristics at above 10 Hz), the Ferry–Porter and Weber laws in the ranges where they apply, the effects of light adaptation, and it accounts for individual differences. By considering the variable RC stage as an approximate analog of a synaptic excitatory process which is controlled by inhibition, significant correspondences are observed between the internal connectivity of the model and the neural connectivity of the retina.

Tradeoffs between stereopsis and proximity luminance covariance as determinants of perceived 3D structure

A 2D polar projection of a 3D wire cube (Necker cube) in clockwise rotation can be perceived either veridically as a clockwise-rotating cube (rigid percept) or as a counterclockwise-rotating rubbery, truncated pyramid (nonrigid percept). The 3D percept is influenced by various cues: linear perspective, stereo disparity, and proximity-luminance covariance (PLC, the intensification of edges in proportion to their proximity to the observer). Perspective, by itself or in combination, is a very weak cue whereas PLC is a powerful cue [Schwartz and Sperling (1983) Bull. Psychon. Soc. 21, 456-458]. Here we determined psychometric functions for perceptual resolution in static displays and dynamic rotating displays (with and without a static preview) as determined by stereopsis and PLC in isolation and with both cues jointly, possibly in conflict. Stereopsis was the dominant cue in static displays and in most dynamic displays. When a static display preceded a dynamic display, it strongly influenced the subsequent dynamic percept. Perceptual resolution in all conditions was accurately described by a winner-take-all model in which the strength of evidence for each percept from different cues is simply algebraically added.

Three stages and two systems of visual processing.

Three stages of visual processing determine how internal noise appears to an external observer: light adaptation, contrast gain control and a postsensory/decision stage. Dark noise occurs prior to adaptation, determines dark-adapted absolute thresholds and mimics stationary external noise. Sensory noise occurs after dark adaptation, determines contrast thresholds for sine gratings and similar stimuli, and mimics external noise that increases with mean luminance. Postsensory noise incorporates perceptual, decision and mnemonic processes. It occurs after contrast-gain control and mimics external noise that increases with stimulus contrast (i.e., multiplicative noise). Dark noise and sensory noise are frequency specific and primarily affect weak signals. Only postsensory noise significantly affects the discriminability of strong signals masked by stimulus noise; postsensory noise has constant power over a wide spatial frequency range in which sensory noise varies enormously. Two parallel perceptual regimes jointly serve human object recognition and motion perception: a first-order linear (Fourier) regime that computes relations directly from stimulus luminance, and a second-order nonlinear (nonFourier) rectifying regime that uses the absolute value (or power) of stimulus contrast. When objects or movements are defined by high spatial frequencies (i.e., texture carrier frequencies whose wavelengths are small compared to the object size), the responses of high-frequency receptors are demodulated by rectification to facilitate discrimination at the higher processing levels. Rectification sacrifices the statistical efficiency (noise resistance) of the first-order regime for efficiency of neural connectivity and computation.

Information transfer in iconic memory experiments

To report letters from briefly exposed letter arrays, subjects must transfer information from a rapidly decaying trace (iconic memory) to more durable storage. In a partial-report paradigm, we systematically varied the proportion (P) of trials with a long cue delay relative to a short cue delay. Practiced subjects used the same transfer strategy independent of P. Data from a partial-report-plus-masking experiment were used to construct a computational model that accurately predicted partial- and whole-report performance with and without masks. Assumptions: Prior to a cue, subjects attend primarily to the middle row of a three-row display, resulting in nonselective transfer. After the cue, they attend only to the cued row. Transfer rate is the product of iconic legibility (which depends on time and retinal location) and attention allocation (which shifts after a cue). Cumulative transfer is limited by the capacity of durable storage.