Tianming Yang

Probabilistic reasoning by neurons

Our brains allow us to reason about alternatives and to make choices that are likely to pay off. Often there is no one correct answer, but instead one that is favoured simply because it is more likely to lead to reward. A variety of probabilistic classification tasks probe the covert strategies that humans use to decide among alternatives based on evidence that bears only probabilistically on outcome. Here we show that rhesus monkeys can also achieve such reasoning. We have trained two monkeys to choose between a pair of coloured targets after viewing four shapes, shown sequentially, that governed the probability that one of the targets would furnish reward. Monkeys learned to combine probabilistic information from the shape combinations. Moreover, neurons in the parietal cortex reveal the addition and subtraction of probabilistic quantities that underlie decision-making on this task.

The Effect of Perceptual Learning on Neuronal Responses in Monkey Visual Area V4

Previous studies have shown that perceptual learning can substantially alter the response properties of neurons in the primary somatosensory and auditory cortices. Although psychophysical studies suggest that perceptual learning induces similar changes in primary visual cortex (V1), studies that have measured the response properties of individual neurons have failed to find effects of the size described for the other sensory systems. We have examined the effect of learning on neuronal response properties in a visual area that lies at a later stage of cortical processing, area V4. Adult macaque monkeys were trained extensively on orientation discrimination at a specific retinal location using a narrow range of orientations. During the course of training, the subjects achieved substantial improvement in orientation discrimination that was primarily restricted to the trained location. After training, neurons in V4 with receptive fields overlapping the trained location had stronger responses and narrower orientation tuning curves than neurons with receptive fields in the opposite, untrained hemifield. The changes were most prominent for neurons that preferred orientations close to the trained range of orientations. These results provide the first demonstration of perceptual learning modifying basic neuronal response properties at an intermediate level of visual cortex and give insights into the distribution of plasticity across adult visual cortex.

A role for primate subgenual cingulate cortex in sustaining autonomic arousal.

Significance Dysregulation of emotion is central to the etiology of mood disorders, such as depression. A causal understanding of how neural structures regulate emotion and arousal could help to improve treatments for these psychiatric disorders. Studies of patients with depression indicate that a particular part of the frontal lobe, the subgenual cingulate cortex, plays an important role in affective processing, though its precise contribution remains unclear. Here we show that, in macaque monkeys, this small part of the frontal cortex is necessary for sustaining elevated arousal in anticipation of positive emotional events. This finding suggests a mechanism for the contribution of this area to affective regulation, including an account for the lack of pleasure and passivity that characterizes mood disorders. The subgenual anterior cingulate cortex (subgenual ACC) plays an important role in regulating emotion, and degeneration in this area correlates with depressed mood and anhedonia. Despite this understanding, it remains unknown how this part of the prefrontal cortex causally contributes to emotion, especially positive emotions. Using Pavlovian conditioning procedures in macaque monkeys, we examined the contribution of the subgenual ACC to autonomic arousal associated with positive emotional events. After such conditioning, autonomic arousal increases in response to cues that predict rewards, and monkeys maintain this heightened state of arousal during an interval before reward delivery. Here we show that although monkeys with lesions of the subgenual ACC show the initial, cue-evoked arousal, they fail to sustain a high level of arousal until the anticipated reward is delivered. Control procedures showed that this impairment did not result from differences in autonomic responses to reward delivery alone, an inability to learn the association between cues and rewards, or to alterations in the light reflex. Our data indicate that the subgenual ACC may contribute to positive affect by sustaining arousal in anticipation of positive emotional events. A failure to maintain positive affect for expected pleasurable events could provide insight into the pathophysiology of psychological disorders in which negative emotions dominate a patient’s affective experience.

Comment on "Single-trial spike trains in parietal cortex reveal discrete steps during decision-making".

Latimer et al. (Reports, 10 July 2015, p. 184) claim that during perceptual decision formation, parietal neurons undergo one-time, discrete steps in firing rate instead of gradual changes that represent the accumulation of evidence. However, that conclusion rests on unsubstantiated assumptions about the time window of evidence accumulation, and their stepping model cannot explain existing data as effectively as evidence-accumulation models.

The strength of gradually accruing probabilistic evidence modulates brain activity during a categorical decision

The evolution of neural activity during a perceptual decision is well characterized by the evidence parameter in sequential sampling models. However, it is not known whether accumulating signals in human neuroimaging are related to the integration of evidence. Our aim was to determine whether activity accumulates in a nonperceptual task by identifying brain regions tracking the strength of probabilistic evidence. fMRI was used to measure whole-brain activity as choices were informed by integrating a series of learned prior probabilities. Participants first learned the predictive relationship between a set of shape stimuli and one of two choices. During scanned testing, they made binary choices informed by the sum of the predictive strengths of individual shapes. Sequences of shapes adhered to three distinct rates of evidence (RoEs): rapid, gradual, and switch. We predicted that activity in regions informing the decision would modulate as a function of RoE prior to the choice. Activity in some regions, including premotor areas, changed as a function of RoE and response hand, indicating a role in forming an intention to respond. Regions in occipital, temporal, and parietal lobes modulated as a function of RoE only, suggesting a preresponse stage of evidence processing. In all of these regions, activity was greatest on rapid trials and least on switch trials, which is consistent with an accumulation-to-boundary account. In contrast, activity in a set of frontal and parietal regions was greatest on switch and least on rapid trials, which is consistent with an effort or time-on-task account.

Contributions of the hippocampus and entorhinal cortex to rapid visuomotor learning in rhesus monkeys.

The hippocampus and adjacent structures in the medial temporal lobe are essential for establishing new associative memories. Despite this knowledge, it is not known whether the hippocampus proper is essential for establishing such memories, nor is it known whether adjacent regions like the entorhinal cortex might contribute. To test the contributions of these regions to the formation of new associative memories, we trained rhesus monkeys to rapidly acquire arbitrary visuomotor associations, i.e., associations between visual stimuli and spatially directed actions. We then assessed the effects of reversible inactivations of either the hippocampus (Experiment 1) or entorhinal cortex (Experiment 2) on the within‐session rate of learning. For comparison, we also evaluated the effects of the inactivations on performance of problems of the same type that had been well learned prior to any inactivations. We found that inactivation of the entorhinal cortex but not hippocampus produced impairments in acquiring novel arbitrary associations. The impairment did not extend to the familiar, previously established associations. These data indicate that the entorhinal cortex is causally involved in establishing new associations, as opposed to retrieving previously learned associations. Published 2014. This article is a U.S. Government work and is in the public domain in the USA.

Effects of Anterior Capsulotomy on Decision Making in Patients with Refractory Obsessive–Compulsive Disorder

Despite various lines of evidence implicating impaired decision-making ability in individuals with obsessive–compulsive disorder (OCD), neuropsychological investigation has generated inconsistent findings. Although the cortico-striato-thalamo-cortical (CSTC) circuitry has been suggested, the involvement of the cortex has not yet been fully demonstrated. Moreover, it is unknown whether surgical intervention on the CSTC circuitry results in a predicted improvement of decision-making ability of OCD. Here we present a study of decision making based on the Iowa Gambling Task (IGT) to investigate decision making in a large sample of individuals with treatment-resistant OCD with and without anterior capsulotomy (AC). Task performance was evaluated in healthy subjects, individuals with OCD that had not undergone surgery, and postsurgical OCD patients with AC. The latter group was further divided into a short-term postsurgical group and a long-term postsurgical group. We found that the OCD patients without surgery performed significantly worse than the healthy controls on the IGT. There were no significant differences in decision-making between the presurgical OCD patients and those at the short-term postsurgical follow-up. Decision-making ability of the long-term postsurgical OCD patients was improved to the level comparable to that of healthy controls. All clinical symptoms (OCD, depression, and anxiety) assessed by psychiatric rating scales were significantly alleviated post-surgically, but exhibited no correlation with their IGT task performance. Our findings provide strong evidence that OCD is linked to impairments in decision-making ability; that impaired CSTC circuitry function is directly involved in the manifestation of OCD; and that AC related improvements in cognitive functions are caused by long-term plasticity in the brain circuitry.

Efficient reinforcement learning of a reservoir network model of parametric working memory achieved with a cluster population winner-take-all readout mechanism

The brain often has to make decisions based on information stored in working memory, but the neural circuitry underlying working memory is not fully understood. Many theoretical efforts have been focused on modeling the persistent delay period activity in the prefrontal areas that is believed to represent working memory. Recent experiments reveal that the delay period activity in the prefrontal cortex is neither static nor homogeneous as previously assumed. Models based on reservoir networks have been proposed to model such a dynamical activity pattern. The connections between neurons within a reservoir are random and do not require explicit tuning. Information storage does not depend on the stable states of the network. However, it is not clear how the encoded information can be retrieved for decision making with a biologically realistic algorithm. We therefore built a reservoir-based neural network to model the neuronal responses of the prefrontal cortex in a somatosensory delayed discrimination task. We first illustrate that the neurons in the reservoir exhibit a heterogeneous and dynamical delay period activity observed in previous experiments. Then we show that a cluster population circuit decodes the information from the reservoir with a winner-take-all mechanism and contributes to the decision making. Finally, we show that the model achieves a good performance rapidly by shaping only the readout with reinforcement learning. Our model reproduces important features of previous behavior and neurophysiology data. We illustrate for the first time how task-specific information stored in a reservoir network can be retrieved with a biologically plausible reinforcement learning training scheme.

Covert shift of attention modulates the value encoding in the orbitofrontal cortex

During value-based decision making, we often evaluate the value of each option sequentially by shifting our attention, even when the options are presented simultaneously. The orbitofrontal cortex (OFC) has been suggested to encode value during value-based decision making. Yet it is not known how its activity is modulated by attention shifts. We investigated this question by employing a passive viewing task that allowed us to disentangle effects of attention, value, choice and eye movement. We found that the attention modulated OFC activity through a winner-take-all mechanism. When we attracted the monkeys’ attention covertly, the OFC neuronal activity reflected the reward value of the newly attended cue. The shift of attention could be explained by a normalization model. Our results strongly argue for the hypothesis that the OFC neuronal activity represents the value of the attended item. They provide important insights toward understanding the OFC’s role in value-based decision making.

A neural network model for the orbitofrontal cortex and task space acquisition during reinforcement learning

Reinforcement learning has been widely used in explaining animal behavior. In reinforcement learning, the agent learns the value of the states in the task, collectively constituting the task state space, and uses the knowledge to choose actions and acquire desired outcomes. It has been proposed that the orbitofrontal cortex (OFC) encodes the task state space during reinforcement learning. However, it is not well understood how the OFC acquires and stores task state information. Here, we propose a neural network model based on reservoir computing. Reservoir networks exhibit heterogeneous and dynamic activity patterns that are suitable to encode task states. The information can be extracted by a linear readout trained with reinforcement learning. We demonstrate how the network acquires and stores task structures. The network exhibits reinforcement learning behavior and its aspects resemble experimental findings of the OFC. Our study provides a theoretical explanation of how the OFC may contribute to reinforcement learning and a new approach to understanding the neural mechanism underlying reinforcement learning.