Carl R. Olson

Statistical learning of visual transitions in monkey inferotemporal cortex

One of the most fundamental functions of the brain is to predict upcoming events on the basis of the recent past. A closely related function is to signal when a prediction has been violated. The identity of the brain regions that mediate these functions is not known. We set out to determine whether they are implemented at the level of single neurons in the visual system. We gave monkeys prolonged exposure to pairs of images presented in fixed sequence so that each leading image became a strong predictor for the corresponding trailing image. We then monitored the responses of neurons in the inferotemporal cortex to image sequences that obeyed or violated the transitional rules imposed during training. Inferotemporal neurons exhibited a transitional surprise effect, responding much more strongly to unpredicted transitions than to predicted transitions. Thus, neurons even in the visual system make experience-based predictions and react when they fail.

In Monkeys Making Value-Based Decisions LIP Neurons Encode Cue Salience and Not Action Value

Salience, Values, and Decisions How does the brain make value-based decisions? There are two major competing models: the goodsbased model and the action-based model of value. Leathers and Olson (p. 132) designed a critical experiment to decide between these two views. In the monkey brain, lateral intraparietal neurons responded strongly to stimuli predicting both large rewards and large penalties, encoding the salience of the stimulus rather than reward value, which refutes both models. Parietal cortex neurons respond to cues that predict large penalties as well as large rewards. In monkeys deciding between alternative saccadic eye movements, lateral intraparietal (LIP) neurons representing each saccade fire at a rate proportional to the value of the reward expected upon its completion. This observation has been interpreted as indicating that LIP neurons encode saccadic value and that they mediate value-based decisions between saccades. Here, we show that LIP neurons representing a given saccade fire strongly not only if it will yield a large reward but also if it will incur a large penalty. This finding indicates that LIP neurons are sensitive to the motivational salience of cues. It is compatible neither with the idea that LIP neurons represent action value nor with the idea that value-based decisions take place in LIP neurons.

Object-based vision and attention in primates

In forming a representation of a visible object, the brain must analyze the visual scene pre-attentively, select an object through active attention, and form representations of the multiple attributes of the selected object. During the past two years, progress has been made in understanding the neural underpinnings of these processes by means of single-neuron recording in monkeys.

Neuronal activity related to anticipated reward in frontal cortex: does it represent value or reflect motivation?

Abstract: It is thought that neuronal activity in orbitofrontal cortex (OFC) represents the value of anticipated reward; however activity in many other brain areas also seems to reflect expected reward value. For example, we have shown that in monkeys performing a memory‐guided saccade task for a reward of variable size, activity in numerous areas of frontal cortex is stronger when the monkey anticipates a larger reward. The activity of these neurons might be related to the value of the expected reward or to the degree of motivation induced by expectation of the reward. Anticipation of a more valued reward leads to stronger motivation, as evidenced by measures of arousal, attention, and intensity of motor output. On the assumption that motivated behavior depends on influences arising in the limbic system and acting on the motor system, we hypothesized that neuronal signals representing reward value are unique to OFC, whereas signals arising from other frontal areas, those more closely tied the motor system, reflect the degree of motivation. To test this hypothesis, we recorded from single neurons in OFC and premotor cortex while two monkeys performed a task in which we dissociated value from motivation. Neuronal activity in premotor cortex reflected the monkeys degree of motivation, presumably related to the monkeys level of motor readiness and movement preparation, whereas neuronal activity in OFC represented the value of expected reward.

Rank Signals in Four Areas of Macaque Frontal Cortex During Selection of Actions and Objects in Serial Order

Neurons in several areas of monkey frontal cortex exhibit ordinal position (rank) selectivity during the performance of serial order tasks. It has been unclear whether rank selectivity or the dependence of rank selectivity on task context varies across the areas of frontal cortex. To resolve this issue, we recorded from neurons in the supplementary motor area (SMA), presupplementary motor area (pre-SMA), supplementary eye field (SEF), and dorsolateral prefrontal cortex (dlPFC) as monkeys performed two oculomotor tasks, one requiring the selection of three actions in sequence and the other requiring the selection of three objects in sequence. We found that neurons representing all ranks were present in all areas. Only to a moderate degree did the prevalence and nature of rank selectivity vary from area to area. The two most prominent inter-area differences involved a lower prevalence of rank selectivity in the dlPFC than in the other areas and a higher proportion of neurons preferring late ranks in the SMA and SEF than in the other areas. Neurons in all four areas are rank generalists in the sense of favoring the same rank in both the serial action task and the serial object task.

Representing the Forest before the Trees: A Global Advantage Effect in Monkey Inferotemporal Cortex

Hierarchical stimuli (large shapes composed of small shapes) have long been used to study how humans perceive the global and the local content of a scene—the forest and the trees. Studies using these stimuli have revealed a global advantage effect: humans consistently report global shape faster than local shape. The neuronal underpinnings of this effect remain unclear. Here we demonstrate a correlate and possible mechanism in monkey inferotemporal cortex (IT). Inferotemporal neurons signal the global content of a hierarchical display ~30 ms before they signal its local content. This is a specific expression of a general principle, related to spatial scale or spatial frequency rather than to hierarchical level, whereby the representation of a large shape develops in IT before that of a small shape. These findings provide support for a coarse-to-fine model of visual scene representation.

Statistical Learning of Serial Visual Transitions by Neurons in Monkey Inferotemporal Cortex

If monkeys repeatedly, over the course of weeks, view displays in which two images appear in fixed sequence, then neurons of inferotemporal cortex (ITC) come to exhibit prediction suppression. The response to the trailing image is weaker if it follows the leading image with which it was paired during training than if it follows some other leading image. Prediction suppression is a plausible neural mechanism for statistical learning of visual transitions such as has been demonstrated in behavioral studies of human infants and adults. However, in the human studies, subjects are exposed to continuous sequences in which the same image can be both predicted and predicting and statistical dependency can exist between nonadjacent items. The aim of the present study was to investigate whether prediction suppression in ITC develops under such circumstances. To resolve this issue, we exposed monkeys repeatedly to triplets of images presented in fixed order. The results indicate that prediction suppression can be induced by training not only with pairs of images but also with longer sequences.

Responses to Compound Objects in Monkey Inferotemporal Cortex: The Whole Is Equal to the Sum of the Discrete Parts

It is commonly thought that neurons in monkey inferotemporal cortex are conjunction selective—that a neuron will respond to an image if and only if it contains a required combination of parts. However, this view is based on the results of experiments manipulating closely adjacent or confluent parts. Neurons may have been sensitive not to the conjunction of parts as such but to the presence of a unique feature created where they abut. Here, we compare responses to two sets of images, one composed of spatially separate and the other of abutting parts. We show that the influences of spatially separate parts combine, to a very close approximation, according to a linear rule. Nonlinearities are more prominent—although still weak—in responses to images composed of abutting parts.

Monkey Supplementary Eye Field Neurons Signal the Ordinal Position of Both Actions and Objects

When a monkey executes a learned series of eye movements (for example, rightward followed by upward followed by leftward), neurons in the supplementary eye field (SEF) fire differentially in conjunction with the first, second, and third movements. It has not been clear whether such ordinal position signals are truly general, accompanying all forms of sequential behavior, or accompany only learned sequences of movements. To resolve this issue, we trained monkeys to perform both a serial action task (making saccades in a fixed sequence of directions) and a serial object task (making saccades to a fixed sequence of objects). We found concordant ordinal position selectivity in the two tasks. Neuronal selectivity for the passage of time and expectation of reward could not explain such concordance. We conclude that SEF neurons signal ordinal position consistently across different task contexts. These signals presumably underlie the ability of primates including humans to perform a broad range of serial order tasks.

Global Image Dissimilarity in Macaque Inferotemporal Cortex Predicts Human Visual Search Efficiency

Finding a target in a visual scene can be easy or difficult depending on the nature of the distractors. Research in humans has suggested that search is more difficult the more similar the target and distractors are to each other. However, it has not yielded an objective definition of similarity. We hypothesized that visual search performance depends on similarity as determined by the degree to which two images elicit overlapping patterns of neuronal activity in visual cortex. To test this idea, we recorded from neurons in monkey inferotemporal cortex (IT) and assessed visual search performance in humans using pairs of images formed from the same local features in different global arrangements. The ability of IT neurons to discriminate between two images was strongly predictive of the ability of humans to discriminate between them during visual search, accounting overall for 90% of the variance in human performance. A simple physical measure of global similarity—the degree of overlap between the coarse footprints of a pair of images—largely explains both the neuronal and the behavioral results. To explain the relation between population activity and search behavior, we propose a model in which the efficiency of global oddball search depends on contrast-enhancing lateral interactions in high-order visual cortex.