Alireza Soltani

Mechanisms underlying cortical activity during value-guided choice.

When choosing between two options, correlates of their value are represented in neural activity throughout the brain. Whether these representations reflect activity that is fundamental to the computational process of value comparison, as opposed to other computations covarying with value, is unknown. We investigated activity in a biophysically plausible network model that transforms inputs relating to value into categorical choices. A set of characteristic time-varying signals emerged that reflect value comparison. We tested these model predictions using magnetoencephalography data recorded from human subjects performing value-guided decisions. Parietal and prefrontal signals matched closely with model predictions. These results provide a mechanistic explanation of neural signals recorded during value-guided choice and a means of distinguishing computational roles of different cortical regions whose activity covaries with value.

A Biophysically Based Neural Model of Matching Law Behavior: Melioration by Stochastic Synapses

In experiments designed to uncover the neural basis of adaptive decision making in a foraging environment, neuroscientists have reported single-cell activities in the lateral intraparietal cortex (LIP) that are correlated with choice options and their subjective values. To investigate the underlying synaptic mechanism, we considered a spiking neuron model of decision making endowed with synaptic plasticity that follows a reward-dependent stochastic Hebbian learning rule. This general model is tested in a matching task in which rewards on two targets are scheduled randomly with different rates. Our main results are threefold. First, we show that plastic synapses provide a natural way to integrate past rewards and estimate the local (in time) “return” of a choice. Second, our model reproduces the matching behavior (i.e., the proportional allocation of choices matches the relative reinforcement obtained on those choices, which is achieved through melioration in individual trials). Our model also explains the observed “undermatching” phenomenon and points to biophysical constraints (such as finite learning rate and stochastic neuronal firing) that set the limits to matching behavior. Third, although our decision model is an attractor network exhibiting winner-take-all competition, it captures graded neural spiking activities observed in LIP, when the latter were sorted according to the choices and the difference in the returns for the two targets. These results suggest that neurons in LIP are involved in selecting the oculomotor responses, whereas rewards are integrated and stored elsewhere, possibly by plastic synapses and in the form of the return rather than income of choice options.

A Range-Normalization Model of Context-Dependent Choice: A New Model and Evidence

Most utility theories of choice assume that the introduction of an irrelevant option (called the decoy) to a choice set does not change the preference between existing options. On the contrary, a wealth of behavioral data demonstrates the dependence of preference on the decoy and on the context in which the options are presented. Nevertheless, neural mechanisms underlying context-dependent preference are poorly understood. In order to shed light on these mechanisms, we design and perform a novel experiment to measure within-subject decoy effects. We find within-subject decoy effects similar to what have been shown previously with between-subject designs. More importantly, we find that not only are the decoy effects correlated, pointing to similar underlying mechanisms, but also these effects increase with the distance of the decoy from the original options. To explain these observations, we construct a plausible neuronal model that can account for decoy effects based on the trial-by-trial adjustment of neural representations to the set of available options. This adjustment mechanism, which we call range normalization, occurs when the nervous system is required to represent different stimuli distinguishably, while being limited to using bounded neural activity. The proposed model captures our experimental observations and makes new predictions about the influence of the choice set size on the decoy effects, which are in contrast to previous models of context-dependent choice preference. Critically, unlike previous psychological models, the computational resource required by our range-normalization model does not increase exponentially as the set size increases. Our results show that context-dependent choice behavior, which is commonly perceived as an irrational response to the presence of irrelevant options, could be a natural consequence of the biophysical limits of neural representation in the brain.

Visual Saliency Computations: Mechanisms, Constraints, and the Effect of Feedback

The primate visual system continuously selects spatial proscribed regions, features or objects for further processing. These selection mechanisms—collectively termed selective visual attention—are guided by intrinsic, bottom-up and by task-dependent, top-down signals. While much psychophysical research has shown that overt and covert attention is partially allocated based on saliency-driven exogenous signals, it is unclear how this is accomplished at the neuronal level. Recent electrophysiological experiments in monkeys point to the gradual emergence of saliency signals when ascending the dorsal visual stream and to the influence of top-down attention on these signals. To elucidate the neural mechanisms underlying these observations, we construct a biologically plausible network of spiking neurons to simulate the formation of saliency signals in different cortical areas. We find that saliency signals are rapidly generated through lateral excitation and inhibition in successive layers of neural populations selective to a single feature. These signals can be improved by feedback from a higher cortical area that represents a saliency map. In addition, we show how top-down attention can affect the saliency signals by disrupting this feedback through its action on the saliency map. While we find that saliency computations require dominant slow NMDA currents, the signal rapidly emerges from successive regions of the network. In conclusion, using a detailed spiking network model we find biophysical mechanisms and limitations of saliency computations which can be tested experimentally.

Belief about nicotine selectively modulates value and reward prediction error signals in smokers

Significance Nicotine is the primary addictive substance in tobacco, which stimulates neural pathways mediating reward processing. However, pure biochemical explanations are not sufficient to account for the difficulty in quitting and remaining smoke-free among smokers, and in fact cognitive factors are now considered to contribute critically to addiction. Using model-based functional neuroimaging, we show that smokers’ prior beliefs about nicotine specifically impact learning signals defined by principled computational models of mesolimbic dopamine systems. We further demonstrate that these specific changes in neural signaling are accompanied by measurable changes in smokers’ choice behavior. Our findings suggest that subjective beliefs can override the physical presence of a powerful drug like nicotine by modulating learning signals processed in the brain’s reward system. Little is known about how prior beliefs impact biophysically described processes in the presence of neuroactive drugs, which presents a profound challenge to the understanding of the mechanisms and treatments of addiction. We engineered smokers’ prior beliefs about the presence of nicotine in a cigarette smoked before a functional magnetic resonance imaging session where subjects carried out a sequential choice task. Using a model-based approach, we show that smokers’ beliefs about nicotine specifically modulated learning signals (value and reward prediction error) defined by a computational model of mesolimbic dopamine systems. Belief of “no nicotine in cigarette” (compared with “nicotine in cigarette”) strongly diminished neural responses in the striatum to value and reward prediction errors and reduced the impact of both on smokers’ choices. These effects of belief could not be explained by global changes in visual attention and were specific to value and reward prediction errors. Thus, by modulating the expression of computationally explicit signals important for valuation and choice, beliefs can override the physical presence of a potent neuroactive compound like nicotine. These selective effects of belief demonstrate that belief can modulate model-based parameters important for learning. The implications of these findings may be far ranging because belief-dependent effects on learning signals could impact a host of other behaviors in addiction as well as in other mental health problems.

From biophysics to cognition: reward-dependent adaptive choice behavior

In neurobiological studies of various cognitive abilities, neuroscientists use mathematical models to fit behavioral data from well-controlled experiments and look for neural activities that are correlated with parameters in those models. The pinpointed neural correlates are often taken as evidence that a given task is performed according to the prescription of the applied model, and the relevant brain areas encode parameters of such a model. However, to go beyond correlations toward causal understanding, it is necessary to elucidate at multiple levels the neural circuit mechanisms of cognitive processes. This review focuses on recent studies of reward-based decision-making that have begun to tackle this challenge.

Dissociable dopaminergic control of saccadic target selection and its implications for reward modulation

To investigate mechanisms by which reward modulates target selection, we studied the behavioral effects of perturbing dopaminergic activity within the frontal eye field (FEF) of monkeys performing a saccadic choice task and simulated the effects using a plausible cortical network. We found that manipulation of FEF activity either by blocking D1 receptors (D1Rs) or by stimulating D2 receptors (D2Rs) increased the tendency to choose targets in the response field of the affected site. However, the D1R manipulation decreased the tendency to repeat choices on subsequent trials, whereas the D2R manipulation increased that tendency. Moreover, the amount of shift in target selection resulting from the two manipulations correlated in opposite ways with the baseline stochasticity of choice behavior. Our network simulation results suggest that D1Rs influence target selection mainly through their effects on the strength of inputs to the FEF and on recurrent connectivity, whereas D2Rs influence the excitability of FEF output neurons. Altogether, these results reveal dissociable dopaminergic mechanisms influencing target selection and suggest how reward can influence adaptive choice behavior via prefrontal dopamine.

Metaplasticity as a Neural Substrate for Adaptive Learning and Choice under Uncertainty

Value-based decision making often involves integration of reward outcomes over time, but this becomes considerably more challenging if reward assignments on alternative options are probabilistic and non-stationary. Despite the existence of various models for optimally integrating reward under uncertainty, the underlying neural mechanisms are still unknown. Here we propose that reward-dependent metaplasticity (RDMP) can provide a plausible mechanism for both integration of reward under uncertainty and estimation of uncertainty itself. We show that a model based on RDMP can robustly perform the probabilistic reversal learning task via dynamic adjustment of learning based on reward feedback, while changes in its activity signal unexpected uncertainty. The model predicts time-dependent and choice-specific learning rates that strongly depend on reward history. Key predictions from this model were confirmed with behavioral data from non-human primates. Overall, our results suggest that metaplasticity can provide a neural substrate for adaptive learning and choice under uncertainty.

Belief about nicotine modulates subjective craving and insula activity in deprived smokers

Little is known about the specific neural mechanisms through which cognitive factors influence craving and associated brain responses, despite the initial success of cognitive therapies in treating drug addiction. In this study, we investigated how cognitive factors such as beliefs influence subjective craving and neural activities in nicotine-addicted individuals using model-based functional magnetic resonance imaging (fMRI) and neuropharmacology. Deprived smokers (N = 24) participated in a two-by-two balanced placebo design, which crossed beliefs about nicotine (told “nicotine” vs. told “no nicotine”) with the nicotine content in a cigarette (nicotine vs. placebo) which participants smoked immediately before performing a fMRI task involving reward learning. Subjects’ reported craving was measured both before smoking and after the fMRI session. We found that first, in the presence of nicotine, smokers demonstrated significantly reduced craving after smoking when told “nicotine in cigarette” but showed no change in craving when told “no nicotine.” Second, neural activity in the insular cortex related to craving was only significant when smokers were told “nicotine” but not when told “no nicotine.” Both effects were absent in the placebo condition. Third, insula activation related to computational learning signals was modulated by belief about nicotine regardless of nicotine’s presence. These results suggest that belief about nicotine has a strong impact on subjective craving and insula responses related to both craving and learning in deprived smokers, providing insights into the complex nature of belief–drug interactions.

Social Saliency of the Cue Slows Attention Shifts

Eye gaze is a powerful cue that indicates where another person’s attention is directed in the environment. Seeing another person’s eye gaze shift spontaneously and reflexively elicits a shift of one’s own attention to the same region in space. Here, we investigated whether reallocation of attention in the direction of eye gaze is modulated by personal familiarity with faces. On the one hand, the eye gaze of a close friend should be more effective in redirecting our attention as compared to the eye gaze of a stranger. On the other hand, the social relevance of a familiar face might itself hold attention and, thereby, slow lateral shifts of attention. To distinguish between these possibilities, we measured the efficacy of the eye gaze of personally familiar and unfamiliar faces as directional attention cues using adapted versions of the Posner paradigm with saccadic and manual responses. We found that attention shifts were slower when elicited by a perceived change in the eye gaze of a familiar individual as compared to attention shifts elicited by unfamiliar faces at short latencies (100 ms). We also measured simple detection of change in direction of gaze in personally familiar and unfamiliar faces to test whether slower attention shifts were due to slower detection. Participants detected changes in eye gaze faster for familiar faces than for unfamiliar faces. Our results suggest that personally familiar faces briefly hold attention due to their social relevance, thereby slowing shifts of attention, even though the direction of eye movements are detected faster in familiar faces.