Robert A. Marino

Free viewing of dynamic stimuli by humans and monkeys

Due to extensive homologies, monkeys provide a sophisticated animal model of human visual attention. However, for electrophysiological recording in behaving animals simplified stimuli and controlled eye position are traditionally used. To validate monkeys as a model for human attention during realistic free viewing, we contrasted human (n = 5) and monkey (n = 5) gaze behavior using 115 natural and artificial video clips. Monkeys exhibited broader ranges of saccadic endpoints and amplitudes and showed differences in fixation and intersaccadic intervals. We compared tendencies of both species to gaze toward scene elements with similar low-level visual attributes using two computational models--luminance contrast and saliency. Saliency was more predictive of both human and monkey gaze, predicting human saccades better than monkey saccades overall. Quantifying interobserver gaze consistency revealed that while humans were highly consistent, monkeys were more heterogeneous and were best predicted by the saliency model. To address these discrepancies, we further analyzed high-interest gaze targets--those locations simultaneously chosen by at least two monkeys. These were on average very similar to human gaze targets, both in terms of specific locations and saliency values. Although substantial quantitative differences were revealed, strong similarities existed between both species, especially when focusing analysis onto high-interest targets.

Color-Related Signals in the Primate Superior Colliculus

Color is important for segmenting objects from backgrounds, which can in turn facilitate visual search in complex scenes. However, brain areas involved in orienting the eyes toward colored stimuli in our environment are not believed to have access to color information. Here, we show that neurons in the intermediate layers of the monkey superior colliculus (SC), a critical structure for the production of saccadic eye movements, can respond to isoluminant color stimuli with the same magnitude as a maximum contrast luminance stimulus. In contrast, neurons from the superficial SC layers showed little color-related activity. Crucially, visual onset latencies were 30–35 ms longer for color, implying that luminance and chrominance information reach the SC through distinct pathways and that the observed color-related activity is not the result of residual luminance signals. Furthermore, these differences in visual onset latency translated directly into differences in saccadic reaction time. The results demonstrate that the saccadic system can signal the presence of chromatic stimuli only one stage from the brainstem premotor circuitry that drives the eyes.

Spatial Relationships of Visuomotor Transformations in the Superior Colliculus Map

The oculomotor system is well understood compared with other motor systems; however, we do not yet know the spatial details of sensory to motor transformations. This study addresses this issue by quantifying the spatial relationships between visual and motor responses in the superior colliculus (SC), a midbrain structure involved in the transformation of visual information into saccadic motor command signals. We collected extracellular single-unit recordings from 150 visual-motor (VM) and 28 motor (M) neurons in two monkeys trained to perform a nonpredictive visually guided saccade task to 110 possible target locations. Motor related discharge was greater than visual related discharge in 94% (141/150) of the VM neurons. Across the population of VM neurons, the mean locations of the peak visual and motor responses were spatially aligned. The visual response fields (RFs) were significantly smaller than and usually contained within the motor RFs. Converting RFs into the SC coordinate system significantly reduced any misalignment between peak visual and motor locations. RF size increased with increasing eccentricity in visual space but remained invariant on the SC map beyond 1 mm of the rostral pole. RF shape was significantly more symmetric in SC map coordinates compared with visual space coordinates. These results demonstrate that VM neurons specify the same location of a target stimulus in the visual field as the intended location of an upcoming saccade with minimal misalignment to downstream structures. The computational consequences of spatially transforming visual field coordinates to the SC map resulted in increased alignment and spatial symmetry during visual-sensory to saccadic-motor transformations.

Spatial interactions in the superior colliculus predict saccade behavior in a neural field model

During natural vision, eye movements are dynamically controlled by the combinations of goal-related top–down (TD) and stimulus-related bottom–up (BU) neural signals that map onto objects or locations of interest in the visual world. In primates, both BU and TD signals converge in many areas of the brain, including the intermediate layers of the superior colliculus (SCi), a midbrain structure that contains a retinotopically coded map for saccades. How TD and BU signals combine or interact within the SCi map to influence saccades remains poorly understood and actively debated. It has been proposed that winner-take-all competition between these signals occurs dynamically within this map to determine the next location for gaze. Here, we examine how TD and BU signals interact spatially within an artificial two-dimensional dynamic winner-take-all neural field model of the SCi to influence saccadic RT (SRT). We measured point images (spatially organized population activity on the SC map) physiologically to inform the TD and BU model parameters. In this model, TD and BU signals interacted nonlinearly within the SCi map to influence SRT via changes to the (1) spatial size or extent of individual signals, (2) peak magnitude of individual signals, (3) total number of competing signals, and (4) the total spatial separation between signals in the visual field. This model reproduced previous behavioral studies of TD and BU influences on SRT and accounted for multiple inconsistencies between them. This is achieved by demonstrating how, under different experimental conditions, the spatial interactions of TD and BU signals can lead to either increases or decreases in SRT. Our results suggest that dynamic winner-take-all modeling with local excitation and distal inhibition in two dimensions accurately reflects both the physiological activity within the SCi map and the behavioral changes in SRT that result from BU and TD manipulations.

Linking visual response properties in the superior colliculus to saccade behavior

Here we examined the influence of the visual response in the superior colliculus (SC) (an oculomotor control structure integrating sensory, motor and cognitive signals) on the development of the motor command that drives saccadic eye movements in monkeys. We varied stimulus luminance to alter the timing and magnitude of visual responses in the SC and examined how these changes correlated with resulting saccade behavior. Increasing target luminance resulted in multiple modulations of the visual response, including increased magnitude and decreased response onset latency. These signal modulations correlated strongly with changes in saccade latency and metrics, indicating that these signal properties carry through to the neural computations that determine when, where and how fast the eyes will move. Thus, components of the earliest part of the visual response in the SC provide important building blocks for the neural basis of the sensory–motor transformation, highlighting a critical link between the properties of the visual response and saccade behavior.

Distinct local circuit properties of the superficial and intermediate layers of the rodent superior colliculus.

The superior colliculus (SC) is critical in localizing salient visual stimuli and making decisions on the location of the next saccade. Lateral interactions across the spatial map of the SC are hypothesized to help mediate these processes. Here, we investigate lateral interactions within the SC by applying whole‐cell recordings in horizontal slices of mouse SC, which maintained the local structure of the superficial (SCs) visual layer, which is hypothesized to participate in localizing salient stimuli, and the intermediate (SCi) layer, which is supposed to participate in saccade decision‐making. When effects of either electrical or chemical (uncaging of free glutamate) stimuli were applied to multiple sites with various distances from the recorded cell, a pattern of center excitation‐surround inhibition was found to be prominent in SCs. When the interactions of synaptic effects induced by simultaneous stimulation of two sites were tested, non‐linear facilitatory or inhibitory interactions were observed. In contrast, in the SCi, stimulation induced mainly excitation, which masked underlying inhibition. The excitatory synaptic effects of stimulation applied at remote sites were summed in a near linear manner. The result suggested that SCs lateral interactions appear suitable for localizing salient stimuli, while the lateral interactions within SCi are more suitable for faithfully accumulating subthreshold signals for saccadic decision‐making. Implementation of this laminar‐specific organization makes the SC a unique structure for serially processing signals for saliency localization and saccadic decision‐making.

Visual adaptation and novelty responses in the superior colliculus

The brain’s ability to ignore repeating, often redundant, information while enhancing novel information processing is paramount to survival. When stimuli are repeatedly presented, the response of visually sensitive neurons decreases in magnitude, that is, neurons adapt or habituate, although the mechanism is not yet known. We monitored the activity of visual neurons in the superior colliculus (SC) of rhesus monkeys who actively fixated while repeated visual events were presented. We dissociated adaptation from habituation as mechanisms of the response decrement by using a Bayesian model of adaptation, and by employing a paradigm including rare trials that included an oddball stimulus that was either brighter or dimmer. If the mechanism is adaptation, response recovery should be seen only for the brighter stimulus; if the mechanism is habituation, response recovery (‘dishabituation’) should be seen for both the brighter and dimmer stimuli. We observed a reduction in the magnitude of the initial transient response and an increase in response onset latency with stimulus repetition for all visually responsive neurons in the SC. Response decrement was successfully captured by the adaptation model, which also predicted the effects of presentation rate and rare luminance changes. However, in a subset of neurons with sustained activity in response to visual stimuli, a novelty signal akin to dishabituation was observed late in the visual response profile for both brighter and dimmer stimuli, and was not captured by the model. This suggests that SC neurons integrate both rapidly discounted information about repeating stimuli and novelty information about oddball events, to support efficient selection in a cluttered dynamic world.

Superior colliculus neurons encode a visual saliency map during free viewing of natural dynamic video

Models of visual attention postulate the existence of a saliency map whose function is to guide attention and gaze to the most conspicuous regions in a visual scene. Although cortical representations of saliency have been reported, there is mounting evidence for a subcortical saliency mechanism, which pre-dates the evolution of neocortex. Here, we conduct a strong test of the saliency hypothesis by comparing the output of a well-established computational saliency model with the activation of neurons in the primate superior colliculus (SC), a midbrain structure associated with attention and gaze, while monkeys watched video of natural scenes. We find that the activity of SC superficial visual-layer neurons (SCs), specifically, is well-predicted by the model. This saliency representation is unlikely to be inherited from fronto-parietal cortices, which do not project to SCs, but may be computed in SCs and relayed to other areas via tectothalamic pathways.

Linking express saccade occurance to stimulus properties and sensorimotor integration in the superior colliculus

Express saccades represent the fastest possible eye movements to visual targets with reaction times that approach minimum sensory-motor conduction delays. Previous work in monkeys has identified two specific neural signals in the superior colliculus (SC: a midbrain sensorimotor integration structure involved in gaze control) that are required to execute express saccades: 1) previsual activity consisting of a low-frequency increase in action potentials in sensory-motor neurons immediately before the arrival of a visual response; and 2) a transient visual-sensory response consisting of a high-frequency burst of action potentials in visually responsive neurons resulting from the appearance of a visual target stimulus. To better understand how these two neural signals interact to produce express saccades, we manipulated the arrival time and magnitude of visual responses in the SC by altering target luminance and we examined the corresponding influences on SC activity and express saccade generation. We recorded from saccade neurons with visual-, motor-, and previsual-related activity in the SC of monkeys performing the gap saccade task while target luminance was systematically varied between 0.001 and 42.5 cd/m(2) against a black background (∼0.0001 cd/m(2)). Our results demonstrated that 1) express saccade latencies were linked directly to the arrival time in the SC of visual responses produced by abruptly appearing visual stimuli; 2) express saccades were generated toward both dim and bright targets whenever sufficient previsual activity was present; and 3) target luminance altered the likelihood of producing an express saccade. When an express saccade was generated, visuomotor neurons increased their activity immediately before the arrival of the visual response in the SC and saccade initiation. Furthermore, the visual and motor responses of visuomotor neurons merged into a single burst of action potentials, while the visual response of visual-only neurons was unaffected. A linear combination model was used to test which SC signals best predicted the likelihood of producing an express saccade. In addition to visual response magnitude and previsual activity of saccade neurons, the model identified presaccadic activity (activity occurring during the 30-ms epoch immediately before saccade initiation) as a third important signal for predicting express saccades. We conclude that express saccades can be predicted by visual, previsual, and presaccadic signals recorded from visuomotor neurons in the intermediate layers of the SC.

Competitive integration of visual and goal-related signals on neuronal accumulation rate: A correlate of oculomotor capture in the superior colliculus

The mechanisms that underlie the integration of visual and goal-related signals for the production of saccades remain poorly understood. Here, we examined how spatial proximity of competing stimuli shapes goal-directed responses in the superior colliculus (SC), a midbrain structure closely associated with the control of visual attention and eye movements. Monkeys were trained to perform an oculomotor-capture task [Theeuwes, J., Kramer, A. F., Hahn, S., Irwin, D. E., & Zelinsky, G. J. Influence of attentional capture on oculomotor control. Journal of Experimental Psychology. Human Perception and Performance, 25, 1595–1608, 1999], in which a target singleton was revealed via an isoluminant color change in all but one item. On a portion of the trials, an additional salient item abruptly appeared near or far from the target. We quantified how spatial proximity between the abrupt-onset and the target shaped the goal-directed response. We found that the appearance of an abrupt-onset near the target induced a transient decrease in goal-directed discharge of SC visuomotor neurons. Although this was indicative of spatial competition, it was immediately followed by a rebound in presaccadic activation, which facilitated the saccadic response (i.e., it induced shorter saccadic RT). A similar suppression also occurred at most nontarget locations even in the absence of the abrupt-onset. This is indicative of a mechanism that enabled monkeys to quickly discount stimuli that shared the common nontarget feature. These results reveal a pattern of excitation/inhibition across the SC visuomotor map that acted to facilitate optimal behavior—the short duration suppression minimized the probability of capture by salient distractors, whereas a subsequent boost in accumulation rate ensured a fast goal-directed response. Such nonlinear dynamics should be incorporated into future biologically plausible models of saccade behavior.