Derek H. Arnold

Spatially Localized Distortions of Event Time

A fundamental question about the perception of time is whether the neural mechanisms underlying temporal judgements are universal and centralized in the brain or modality specific and distributed. Time perception has traditionally been thought to be entirely dissociated from spatial vision. Here we show that the apparent duration of a dynamic stimulus can be manipulated in a local region of visual space by adapting to oscillatory motion or flicker. This implicates spatially localized temporal mechanisms in duration perception. We do not see concomitant changes in the time of onset or offset of the test patterns, demonstrating a direct local effect on duration perception rather than an indirect effect on the time course of neural processing. The effects of adaptation on duration perception can also be dissociated from motion or flicker perception per se. Although 20 Hz adaptation reduces both the apparent temporal frequency and duration of a 10 Hz test stimulus, 5 Hz adaptation increases apparent temporal frequency but has little effect on duration perception. We conclude that there is a peripheral, spatially localized, essentially visual component involved in sensing the duration of visual events.

Asynchronous processing in vision: color leads motion.

It has been demonstrated that subjects do not report changes in color and direction of motion as being co-incidental when they occur synchronously. Instead, for the changes to be reported as being synchronous, changes in direction of motion must precede changes in color. To explain this observation, some researchers have suggested that the neural processing of color and motion is asynchronous. This interpretation has been criticized on the basis that processing time may not correlate directly and invariantly with perceived time of occurrence. Here we examine this possibility by making use of the color-contingent motion aftereffect. By correlating color states disproportionately with two directions of motion, we produced and measured color-contingent motion aftereffects as a function of the range of physical correlations. The aftereffects observed are consistent with the perceptual correlation between color and motion being different from the physical correlation. These findings demonstrate asynchronous processing for different stimulus attributes, with color being processed more quickly than motion. This suggests that the time course of perceptual experience correlates directly with that of neural activity.

Orthogonal adaptation improves orientation discrimination

We investigated the effect of adaptation on orientation discrimination using two experienced observers, then replicated the main effects using a total of 50 naïve subjects. Orientation discrimination around vertical improved after adaptation to either horizontal or vertical gratings, but was impaired by adaptation at 7.5 or 15 degrees from vertical. Improvement was greatest when adapter and test were orthogonal. We show that the results can be understood in terms of a functional model of adaptation in cortical vision.

Shifts of criteria or neural timing? The assumptions underlying timing perception studies

In timing perception studies, the timing of one event is usually manipulated relative to another, and participants are asked to judge if the two events were synchronous, or to judge which of the two events occurred first. Responses are analyzed to determine a measure of central tendency, which is taken as an estimate of the timing at which the two events are perceptually synchronous. When these estimates do not coincide with physical synchrony, it is often assumed that the sensory signals are asynchronous, as though the transfer of information concerning one input has been accelerated or decelerated relative to the other. Here we show that, while this is a viable interpretation, it is equally plausible that such effects are driven by shifts in the criteria used to differentiate simultaneous from asynchronous inputs. Our analyses expose important ambiguities concerning the interpretation of simultaneity judgement data, which have hitherto been underappreciated.

Visual search for a target changing in synchrony with an auditory signal

We examined whether the detection of audio–visual temporal synchrony is determined by a pre-attentive parallel process, or by an attentive serial process using a visual search paradigm. We found that detection of a visual target that changed in synchrony with an auditory stimulus was gradually impaired as the number of unsynchronized visual distractors increased (experiment 1), whereas synchrony discrimination of an attended target in a pre-cued location was unaffected by the presence of distractors (experiment 2). The effect of distractors cannot be ascribed to reduced target visibility nor can the increase in false alarm rates be predicted by a noisy parallel processing model. Reaction times for target detection increased linearly with number of distractors, with the slope being about twice as steep for target-absent trials as for target-present trials (experiment 3). Similar results were obtained regardless of whether the audio–visual stimulus consisted of visual flashes synchronized with amplitude-modulated pips, or of visual rotations synchronized with frequency-modulated up–down sweeps. All of the results indicate that audio–visual perceptual synchrony is judged by a serial process and are consistent with the suggestion that audio–visual temporal synchrony is detected by a ‘mid-level’ feature matching process.

Determinants of asynchronous processing in vision

When a stimulus oscillates in both colour and direction of motion, changes in colour must lag behind those in direction if they are to be seen as concurrent. It has been argued that this lag is the consequence of asynchronous visual processing, with colour being processed more rapidly than motion. This proposal is contentious: it has been criticized on the basis that the time‐course of cortical activity may not correlate directly with that of perceptual experience. Here, we demonstrate that the extent of the apparent asynchrony can vary according to the prevailing stimulus conditions. The apparent asynchrony is greatest if the stimulus is composed of opponent directions of motion and is reduced if the angular difference between the directions is reduced. This pattern of results suggests that asynchronous neural activity arises, in part, as a consequence of differential levels of inhibition within relatively independent cortical structures.

Timing sight and sound

It has been proposed that there is a perceptual compensation for the difference between the speeds of light and sound. We examined this possibility using a range of auditory-visual tasks, in which performance depends on the relative timing of auditory and visual information, and manipulated viewing distance to test for perceptual compensation. We explored auditory-visual integration, cross modal causal attributions, and auditory-visual temporal order judgments. We observed timing shifts with viewing distance following loudspeaker, but not headphone, presentations. We were unable to find reliable evidence of perceptual compensation. Our findings suggest that auditory and visual signals of an event that reach an observer at the same point in time tend to become perceptually bound, even when the sources of those signals could not have occurred together.

Twice Upon a Time Multiple Concurrent Temporal Recalibrations of Audiovisual Speech

Audiovisual timing perception can recalibrate following prolonged exposure to asynchronous auditory and visual inputs. It has been suggested that this might contribute to achieving perceptual synchrony for auditory and visual signals despite differences in physical and neural signal times for sight and sound. However, given that people can be concurrently exposed to multiple audiovisual stimuli with variable neural signal times, a mechanism that recalibrates all audiovisual timing percepts to a single timing relationship could be dysfunctional. In the experiments reported here, we showed that audiovisual temporal recalibration can be specific for particular audiovisual pairings. Participants were shown alternating movies of male and female actors containing positive and negative temporal asynchronies between the auditory and visual streams. We found that audiovisual synchrony estimates for each actor were shifted toward the preceding audiovisual timing relationship for that actor and that such temporal recalibrations occurred in positive and negative directions concurrently. Our results show that humans can form multiple concurrent estimates of appropriate timing for audiovisual synchrony.

A paradox of temporal perception revealed by a stimulus oscillating in colour and orientation

Psychophysical experiments with stimuli oscillating concurrently in colour and orientation revealed an apparently paradoxical dissociation between the perceived simultaneity of stimulus changes and the perceptual pairing of the events demarked by those changes. When subjects were required to report whether changes in colour and orientation were simultaneous, judgements were generally accurate within +/-10 ms. When subjects were required to report which colour was paired predominantly with which orientation, judgements showed a systematic temporal bias of up to 50 ms in favour of colour. This dissociation between different temporal judgements concerning the same stimulus sequence is not predicted by any of the current models of binding in conscious vision. We propose an account of these data based on the temporal response properties of colour- and orientation-selective model neurons such that the perceived pairing of visual attributes is modelled as the cross-correlation of time-varying neural response profiles and thus reflects both neuronal latencies and the rate of rapid adaptation rather than simply the temporal pattern of responses to stimulus transitions.

Perceptual pairing of colour and motion

Observers often pair colours with earlier periods of motion. This observation has prompted the proposal that changes in colour are processed faster and perceived as occurring before physically coincident changes in direction--a brain-time account. Alternatively, it has been proposed that the sudden onset of a surface, or a direction reversal within a persistent surface, can trigger an analysis that determines the perceptual properties of the surface. Hypothetically, this analysis persists for some period of time and the consequences are perceived as having occurred when the analysis commenced--a post-dictive account. Hypotheses based upon these alternate accounts are contrasted in a series of experiments. It is shown that the optimal conditions for pairing specific combinations of colour and motion arise when colour changes are delayed relative to direction changes. In these conditions observers can pair more rapid oscillations of colour and motion and perceptual pairings are more systematic relative to when the changes in colour and direction are physically synchronous. It is also shown that, when pairing colour and motion, the sudden onset of a moving surface does not have the same consequences as a direction reversal within a persistent surface. These findings are consistent with the brain-time, but are inconsistent with the post-dictive, account of perceptual asynchrony.