Paul Apicella

Responses to reward in monkey dorsal and ventral striatum.

SummaryThe sources of input and the behavioral effects of lesions and drug administration suggest that the striatum participates in motivational processes. We investigated the activity of single striatal neurons of monkeys in response to reward delivered for performing in a go-nogo task. A drop of liquid was given each time the animal correctly executed or withheld an arm movement in reaction to a visual stimulus. Of 1593 neurons, 115 showed increased activity in response to delivery of liquid reward in both go and nogo trials. Responding neurons were predominantly located in dorsal and ventromedial parts of anterior putamen, in dorsal and ventral caudate, and in nucleus accumbens. They were twice as frequent in ventral as compared to dorsal striatal areas. Responses occurred at a median latency of 337 ms and lasted for 525 ms, with insignificant differences between dorsal and ventral striatum. Reward responses differed from activity recorded in the face area of posterior putamen which varied synchronously with individual mouth movements. Responses were directly related to delivery of primary liquid reward and not to auditory stimuli associated with it. Most of them also occurred when reward was delivered outside of the task. These results demonstrate that neurons of dorsal and particularly ventral striatum are involved in processing information concerning the attribution of primary reward.

Tonically discharging neurons of monkey striatum respond to preparatory and rewarding stimuli.

SummaryThe behavioral relationships of 396 striatum neurons with regular, tonically elevated discharge rates were studied. While monkeys performed a delayed gonogo task, neurons predominantly located in medial putamen responded with phasic depressions (n = 30) or activations (n = 5) to task-specific stimuli. Particularly effective was an instruction light preparing for movement or no-movement reactions, and an auditory signal associated with reward delivery. Stimuli triggering arm or mouth movements were less effective. The data demonstrate that these usually poorly modulated neurons display context-dependent phasic activity in specific behavioral situations.

Responses of tonically discharging neurons in the monkey striatum to primary rewards delivered during different behavioral states

Abstract In the primate striatum, the tonically discharging neurons respond to conditioned stimuli associated with reward. We investigated whether these neurons respond to the reward itself and how changes in the behavioral context in which the reward is delivered might influence their responsiveness. A total of 286 neurons in the caudate nucleus and putamen were studied in two awake macaque monkeys while liquid reward was delivered in three behavioral situations: (1) an instrumental task, in which reward was delivered upon execution of a visually triggered arm movement; (2) a classically conditioned task, in which reward was delivered 1 s after a visual signal; (3) a free reward situation, in which reward was delivered at irregular time intervals outside of any conditioning task. The monkeys′ uncertainty about the time at which reward will be delivered was assessed by monitoring their mouth movements. A larger proportion of neurons responsive to reward was observed in the free reward situation (86%) than in the classically conditioned (57%) and instrumental tasks (37%). Among the neurons tested in all situations (n = 78), 24% responded to reward regardless of the situation and 65% in only one or two situations. Responses selective for one particular situation occurred exclusively in the free reward situation. When the reward was delivered immediately after the visual signal in the classically conditioned task, most of the neurons reduced or completely lost their responses to reward, and other neurons remained responsive. Conversely, neuronal responses invariably persisted when reward was delivered later than 1 s after the visual signal. This is the first report that tonic striatal neurons might display responses directly to primary rewards. The neuronal responses were strongly influenced by the behavioral context in which the animals received the reward. An important factor appears to be the timing of reward. These neurons might therefore contribute to a general aspect of behavioral reactivity of the subject to relevant stimuli.

Tonically active neurons in the primate striatum and their role in the processing of information about motivationally relevant events

Analysis of recordings of single neuronal activity in the striatum of monkeys engaged in behavioural tasks has shown that tonically active neurons (TANs) can be distinguished by their distinct spontaneous firing and functional properties. As TANs are assumed to be cholinergic interneurons, the study of their physiological characteristics allows us to gain an insight into the role of a particular type of local‐circuit neuron in the processing of information at the striatal level. In monkeys performing various behavioural tasks, the change in the activity of TANs, unlike the diversity of task‐related activations exhibited by the phasically active population of striatal neurons, involves a transient depression of the tonic firing related to environmental events of motivational significance. Such events include primary rewards and stimuli that have acquired a reward value during associative learning. These neurons also respond to an aversive air puff, indicating that their responsiveness is not restricted to appetitive conditions. Another striking feature of the TANs is that their responses can be modulated by predictions about stimulus timing. Temporal variations in event occurrence have been found to favour the responses of TANs, whereas the responses are diminished or abolished in the presence of external cues that predict the time at which events will occur. These data suggest that the TANs respond as do detectors of motivationally relevant events, but they also demonstrate that these neurons are influenced by predictive information based on past experience with a given temporal context. TANs represent a unique subset of striatal neurons that might serve a modulatory function, monitoring for temporal relationships between environmental events.

Responses of monkey midbrain dopamine neurons during delayed alternation performance

Cognitive deficits are important components of the parkinsonian syndrome. In order to investigate the role of dopamine (DA) neurons in cognitive functions, we recorded the electrical activity of midbrain DA neurons in a monkey performing in a spatial delayed alternation task. Triggered by a light, the animal reached toward one of two levers to receive a drop of liquid reward. The lever associated with reward was alternated after each correct movement. Of 88 DA neurons, 65% and 52% showed phasic responses to the trigger light and reward, respectively. By contrast, sustained delay-related activity described for striatum and frontal cortex was not observed, suggesting that the activity of DA neurons does not reflect mnemonic or preparatory representational task components. Rather, DA neurons respond to the salient attentional and motivating stimuli guiding task performance.

Chapter 15 Reward-related activity in the monkey striatum and substantia nigra

Publisher Summary The basal ganglia are often considered as parts of the motor system. This concept is derived from the movement disorders arising in Parkinsonism, choreaand hemiballism, from the projections of the primary motor and somatosensory cortex to the putamen and subthalamic nucleus, and from movement-related neuronal activity in several basal ganglia nuclei. The motor role of the basal ganglia can be extended to include preparatory activity preceding the execution of limb and eye movements, which reflects neuronal access to stored information about forthcoming events and thus constitutes a higher function than primary movement processes. However, there are a number of indications suggesting a larger involvement of the basal ganglia in behavioral processes. In particular, the basal ganglia, notably the striatum, receive input from several limbic structures, such as the amygdala and the orbitofrontal and cingulate cortex. In addition, results from lesioning and psychopharmacological experiments indicate that the functions of the nigrostriatal, mesolimbic and mesocortical dopamine systems encompass a larger spectrum of behavioral processes than a primary motor role suggested by the movement disorders. In particular, they appear to participate in motivational processes determining behavioral activity. In order to assess the involvement of the basal ganglia in motivational processes, this chapter discusses the activity of neurons in the dorsal and ventral striatum and of dopamine neurons in the substantia nigra in relation to the delivery of primary rewards. Examples of primary rewards are food objects or fluids that are approached by subjects through innate or instinctive behavior, or are learned very early during ontogenesis. Primary rewards may serve to establish and sustain learned appetitive behavior, in which case they are called positive reinforcers. Thus, primary rewards are key components in the motivational control of behavior.

Tonically active neurons in the monkey striatum do not preferentially respond to appetitive stimuli.

Abstract The tonically active neurons in the monkey striatum respond to stimuli presented during the performance of appetitively motivated behavior. To test whether these neurons are selectively responsive to the appetitive properties of stimuli, we studied their responsiveness to three different stimuli presented in an unsignalled manner to monkeys not performing any behavioral tasks: (1) an appetitive liquid, eliciting licking movements; (2) an aversive air puff directed towards the face, eliciting eyelid closure and facial movements; (3) a neutral sound, eliciting no overt behavioral reactions. The great majority of the tonic striatal neurons tested in two monkeys showed pronounced responses to the delivery of liquid (338 of 388 neurons, 87%) or the onset of the air puff stimulus (168 of 204, 82%). In contrast, few neurons (15 of 68, 22%) were modulated by the sound. The majority (80%) of the neurons tested with appetitive and aversive stimuli (n=189) responded to both types of stimulus. The characteristics of neuronal responses to the liquid were generally not similar to those described for the air puff in terms of response pattern and response duration. This suggests the existence of differences in the encoding of the affective significance of stimuli. It is concluded that tonic striatal neurons might function to differentiate stimuli that are important to the animal from those that are not, regardless of the specific motivational attributes of relevant stimuli.

Reward-related neuronal activity in the subthalamic nucleus of the monkey

The subthalamic nucleus is a key structure for motor information processing in the basal ganglia. Little is known about its involvement in other aspects of behavior such as motivation. We investigated neuronal activity in the subthalamic nucleus while a monkey performed arm-reaching movements to obtain a liquid reward. Most neurons were modulated both during the movement and reward phases of the task. The changes in activity occurring after or just before the delivery of reward consisted of either increases or decreases in firing and were not directly related to mouth movements. These findings indicate that STN neurons are involved in the detection and expectation of reward, consistent with a role for these neurons in the processing of motivational information.

Cortical and Thalamic Excitation Mediate the Multiphasic Responses of Striatal Cholinergic Interneurons to Motivationally Salient Stimuli

Cholinergic interneurons are key components of striatal microcircuits. In primates, tonically active neurons (putative cholinergic interneurons) exhibit multiphasic responses to motivationally salient stimuli that mirror those of midbrain dopamine neurons and together these two systems mediate reward-related learning in basal ganglia circuits. Here, we addressed the potential contribution of cortical and thalamic excitatory inputs to the characteristic multiphasic responses of cholinergic interneurons in vivo. We first recorded and labeled individual cholinergic interneurons in anesthetized rats. Electron microscopic analyses of these labeled neurons demonstrated that an individual interneuron could form synapses with cortical and, more commonly, thalamic afferents. Single-pulse electrical stimulation of ipsilateral frontal cortex led to robust short-latency (<20 ms) interneuron spiking, indicating monosynaptic connectivity, but firing probability progressively decreased during high-frequency pulse trains. In contrast, single-pulse thalamic stimulation led to weak short-latency spiking, but firing probability increased during pulse trains. After initial excitation from cortex or thalamus, interneurons displayed a “pause” in firing, followed by a “rebound” increase in firing rate. Across all stimulation protocols, the number of spikes in the initial excitation correlated positively with pause duration and negatively with rebound magnitude. The magnitude of the initial excitation, therefore, partly determined the profile of later components of multiphasic responses. Upon examining the responses of tonically active neurons in behaving primates, we found that these correlations held true for unit responses to a reward-predicting stimulus, but not to the reward alone, delivered outside of any task. We conclude that excitatory inputs determine, at least in part, the multiphasic responses of cholinergic interneurons under specific behavioral conditions.

The Role of Striatal Tonically Active Neurons in Reward Prediction Error Signaling during Instrumental Task Performance

The detection of differences between predictions and actual outcomes is important for associative learning and for selecting actions according to their potential future reward. There are reports that tonically active neurons (TANs) in the primate striatum may carry information about errors in the prediction of rewards. However, this property seems to be expressed in classical conditioning tasks but not during performance of an instrumental task. To address this issue, we recorded the activity of TANs in the putamen of two monkeys performing an instrumental task in which probabilistic rewarding outcomes were contingent on an action in block-design experiments. Behavioral evidence suggests that animals adjusted their performance according to the level of probability for reward on each trial block. We found that the TAN response to reward was stronger as the reward probability decreased; this effect was especially prominent on the late component of the pause–rebound pattern of typical response seen in these neurons. The responsiveness to reward omission was also increased with increasing reward probability, whereas there were no detectable effects on responses to the stimulus that triggered the movement. Overall, the modulation of TAN responses by reward probability appeared relatively weak compared with that observed previously in a probabilistic classical conditioning task using the same block design. These data indicate that instrumental conditioning was less effective at demonstrating prediction error signaling in TANs. We conclude that the sensitivity of the TAN system to reward probability depends on the specific learning situation in which animals experienced the stimulus–reward associations.