Gerhard Jocham

Neurophysiology of Performance Monitoring and Adaptive Behavior

Successful goal-directed behavior requires not only correct action selection, planning, and execution but also the ability to flexibly adapt behavior when performance problems occur or the environment changes. A prerequisite for determining the necessity, type, and magnitude of adjustments is to continuously monitor the course and outcome of ones actions. Feedback-control loops correcting deviations from intended states constitute a basic functional principle of adaptation at all levels of the nervous system. Here, we review the neurophysiology of evaluating action course and outcome with respect to their valence, i.e., reward and punishment, and initiating short- and long-term adaptations, learning, and decisions. Based on studies in humans and other mammals, we outline the physiological principles of performance monitoring and subsequent cognitive, motivational, autonomic, and behavioral adaptation and link them to the underlying neuroanatomy, neurochemistry, psychological theories, and computational models. We provide an overview of invasive and noninvasive systemic measures, such as electrophysiological, neuroimaging, and lesion data. We describe how a wide network of brain areas encompassing frontal cortices, basal ganglia, thalamus, and monoaminergic brain stem nuclei detects and evaluates deviations of actual from predicted states indicating changed action costs or outcomes. This information is used to learn and update stimulus and action values, guide action selection, and recruit adaptive mechanisms that compensate errors and optimize goal achievement.

Neuropharmacology of performance monitoring

Adaptive, goal-directed behavior requires that organisms evaluate their actions in terms of their outcomes. Neuroimaging studies show that unfavorable outcomes or situations with high level of conflict engage the posterior medial frontal cortex (pMFC). Recording of event-related potentials revealed that these situations are accompanied by a negative deflection, the so-called error-related negativity (ERN), which appears after an erroneous response or after negative feedback. Both activation of the pMFC and the ERN are thought to represent a signal that indicates the need for behavioral adjustment, and to recruit other brain regions that implement these adjustments. While many fMRI and EEG studies have shed light on the anatomical structures and the cognitive processes involved in performance monitoring, only very recently have researchers begun to investigate the underlying neurochemical mechanisms. Drawing on the putative involvement of dopamine (DA) neurons in coding a reward prediction error, an influential theory has ascribed a pivotal role to DA in performance monitoring. However, although important, DA is certainly not the only neuromodulator involved. Recent studies point to a role for serotonin, norepinephrine and GABA, but also for adenosine in performance monitoring. Here, we review the evidence for neurotransmitter effects on this function in humans. In this light, we critically discuss currently debated models of performance monitoring and potential alternatives.

Dopamine DRD2 Polymorphism Alters Reversal Learning and Associated Neural Activity

In humans, presence of an A1 allele of the DRD2/ANKK1-TaqIa polymorphism is associated with reduced expression of dopamine (DA) D2 receptors in the striatum. Recently, it was observed that carriers of the A1 allele (A1+ subjects) showed impaired learning from negative feedback in a reinforcement learning task. Here, using functional MRI (fMRI), we investigated carriers and noncarriers of the A1 allele while they performed a probabilistic reversal learning task. A1+ subjects showed subtle deficits in reversal learning. In particular, these deficits consisted of an impairment in sustaining the newly rewarded response after a reversal and in a generally decreased tendency to stick with a rewarded response. Both genetic groups showed increased fMRI signal in response to negative feedback in the rostral cingulate zone (RCZ) and anterior insula. Negative feedback that incurred a change in behavior additionally engaged the ventral striatum and a region of the midbrain consistent with the location of dopaminergic cell groups. The response of the RCZ to negative feedback increased as a function of preceding negative feedback. However, this graded response was not observed in the A1+ group. Furthermore, the A1+ group also showed diminished recruitment of the right ventral striatum and the right lateral orbitofrontal cortex (lOFC) during reversals. Together, these results suggest that a genetically driven reduction in DA D2 receptors leads to deficient feedback integration in RCZ. This, in turn, was accompanied by impaired recruitment of the ventral striatum and the right lOFC during reversals, which might explain the behavioral differences between the genetic groups.

Adaptive coding of action values in the human rostral cingulate zone

Correctly selecting appropriate actions in an uncertain environment requires gathering experience about the available actions by sampling them over several trials. Recent findings suggest that the human rostral cingulate zone (RCZ) is important for the integration of extended action–outcome associations across multiple trials and in coding the subjective value of each action. During functional magnetic resonance imaging, healthy volunteers performed two versions of a probabilistic reversal learning task with high (HP) or low (LP) reward probabilities that required them to integrate action–outcome relations over lower or higher numbers of trials, respectively. In the HP session, subjects needed fewer trials to adjust their behavior in response to a reversal of response–reward contingencies. Similarly, the learning rate derived from a reinforcement learning model was higher in the HP condition. This was accompanied by a stronger response of the RCZ to negative feedback upon reversals in the HP condition. Furthermore, RCZ activity related to negative reward prediction errors varied as a function of the learning rate, which determines to what extent the prediction error is used to update action values. These data show that RCZ responses vary as a function of the information content provided by the environment. The more likely a negative event indicates the need for behavioral adaptations, the more prominent is the response of the RCZ. Thus, both the window of trials over which reinforcement information is integrated and adjustment of action values in the RCZ covary with the stochastics of the environment.

Aged and adult rats compared in acquisition and extinction of escape from the water maze: focus on individual differences.

Individual differences in water maze and open-field performance of aged and adult rats were compared in a cross-sectional study. Three- and 24-month-old rats were classified into superior, moderate, and inferior groups on the basis of escape latencies during hidden platform acquisition and were compared regarding water maze acquisition and extinction, and open-field behavior. Unexpectedly, subgroup differences were invariant across age: The inferior and superior maze learners differed in (a) thigmotactic swimming during water maze acquisition and extinction and (b) open-field rearings. Thus, although aging has a detrimental effect on water maze acquisition and extinction, the degree of impairment might be partly determined by individual novelty-induced rearing activity and thigmotactic swimming at adult ages.

Continuous theta-burst stimulation (cTBS) over the lateral prefrontal cortex alters reinforcement learning bias

The prefrontal cortex is known to play a key role in higher-order cognitive functions. Recently, we showed that this brain region is active in reinforcement learning, during which subjects constantly have to integrate trial outcomes in order to optimize performance. To further elucidate the role of the dorsolateral prefrontal cortex (DLPFC) in reinforcement learning, we applied continuous theta-burst stimulation (cTBS) either to the left or right DLPFC, or to the vertex as a control region, respectively, prior to the performance of a probabilistic learning task in an fMRI environment. While there was no influence of cTBS on learning performance per se, we observed a stimulation-dependent modulation of reward vs. punishment sensitivity: Left-hemispherical DLPFC stimulation led to a more reward-guided performance, while right-hemispherical cTBS induced a more avoidance-guided behavior. FMRI results showed enhanced prediction error coding in the ventral striatum in subjects stimulated over the left as compared to the right DLPFC. Both behavioral and imaging results are in line with recent findings that left, but not right-hemispherical stimulation can trigger a release of dopamine in the ventral striatum, which has been suggested to increase the relative impact of rewards rather than punishment on behavior.

Neurokinin(3) receptor antagonism attenuates cocaine's behavioural activating effects yet potentiates its dopamine-enhancing action in the nucleus accumbens core

Several lines of evidence indicate a role for neurokinin3 receptors (NK3‐Rs) in behavioural activation and mechanisms governing reinforcement processes. In this study we investigated the effect of pretreatment with the NK3‐R antagonist, SR142801, (0.2 and 2.0 mg/kg) on the cocaine‐induced (10.0 mg/kg i.p.) increase in extracellular dopaminergic activity in the nucleus accumbens (NAc). In vivo microdialysis in the NAc of freely moving rats showed that cocaine increased concentrations of dopamine (DA) to ∼350% in the core and ∼450% in the shell. Pre‐treatment with SR142801 significantly potentiated this effect in the core (to ∼550%), whereas this effect was not found in the shell. We also investigated the effects of NK3‐Rs antagonism on cocaine‐induced hyperactivity and conditioned place preference. SR142801 blocked the hyperactivity, but neither the conditioned place preference nor the conditioned locomotor activity induced by cocaine, although there was a slight tendency towards a reduced place preference. When given alone, SR142801 had no effects on behaviour or extracellular dopamine concentrations in any of the structures investigated. These data provide evidence for a contribution of NK3‐Rs in the acute behavioural and neurochemical effects of cocaine, involving dopaminergic activity in the core of the nucleus accumbens.

Differential Modulation of Reinforcement Learning by D2 Dopamine and NMDA Glutamate Receptor Antagonism

The firing pattern of midbrain dopamine (DA) neurons is well known to reflect reward prediction errors (PEs), the difference between obtained and expected rewards. The PE is thought to be a crucial signal for instrumental learning, and interference with DA transmission impairs learning. Phasic increases of DA neuron firing during positive PEs are driven by activation of NMDA receptors, whereas phasic suppression of firing during negative PEs is likely mediated by inputs from the lateral habenula. We aimed to determine the contribution of DA D2-class and NMDA receptors to appetitively and aversively motivated reinforcement learning. Healthy human volunteers were scanned with functional magnetic resonance imaging while they performed an instrumental learning task under the influence of either the DA D2 receptor antagonist amisulpride (400 mg), the NMDA receptor antagonist memantine (20 mg), or placebo. Participants quickly learned to select (“approach”) rewarding and to reject (“avoid”) punishing options. Amisulpride impaired both approach and avoidance learning, while memantine mildly attenuated approach learning but had no effect on avoidance learning. These behavioral effects of the antagonists were paralleled by their modulation of striatal PEs. Amisulpride reduced both appetitive and aversive PEs, while memantine diminished appetitive, but not aversive PEs. These data suggest that striatal D2-class receptors contribute to both approach and avoidance learning by detecting both the phasic DA increases and decreases during appetitive and aversive PEs. NMDA receptors on the contrary appear to be required only for approach learning because phasic DA increases during positive PEs are NMDA dependent, whereas phasic decreases during negative PEs are not.

Neurokinin-1 receptor antagonism by SR140333: enhanced in vivo ACh in the hippocampus and promnestic post-trial effects.

Substance P (SP) has memory-promoting, reinforcing and anxiolytic-like effects when applied systemically or centrally. Such effects may be mediated by the neurokinin-1 (NK-1) receptor, since SP preferentially binds to this receptor. We measured the effects of a selective non-peptide NK-1 receptor antagonist, SR140333 (1, 3 and 9 mg/kg i.p.) on ACh levels in frontal cortex, amygdala and hippocampus by microdialysis and HPLC. Levels of ACh in the hippocampus increased dose-dependently immediately after treatment. The same doses of SR140333 given post-trial had minor facilitative effects on inhibitory avoidance learning and open-field habituation, but did not have reinforcing effects in a conditioned place preference (CPP) task. The selective action of NK-1 receptor antagonism on hippocampal ACh may be related to its positive influence on learning.

Interaction of the tachykinin NK3 receptor agonist senktide with behavioral effects of cocaine in marmosets (Callithrix penicillata)

Brain neuropeptide transmitters of the tachykinin family are involved in the organization of many behaviors. However, little is known about their contribution to the behavioral effects of drugs of abuse. Recently, antagonism of the tachykinin NK3-receptor (NK3-R), one of the three tachykinin receptors in the brain, was shown to attenuate the acute and chronic behavioral effects of cocaine in rats and the acute effects in non-human primates. In order to expand these findings we investigated the effects of the NK3-R agonist, succinyl-[Asp6, Me-Phe8]SP(6-11) (senktide), on the acute behavioral effects of cocaine in marmoset monkeys (Callithrix penicillata) using a figure-eight maze procedure. Animals were pretreated with senktide (0, 0.1, 0.2, 0.4 mg/kg, s.c.), and received either a treatment with cocaine (10 mg/kg) or saline (i.p.). Cocaine increased locomotor activity and the duration of aerial scanning behavior, but reduced exploratory activity, bodycare activity, the frequency of aerial scanning, and terrestrial glance behavior. Senktide blocked the effects of cocaine on locomotor activity, but enhanced the cocaine effects on exploratory activity, aerial scanning frequency, and terrestrial glance behavior. Senktide alone did not significantly influence monkey behavior in this study. These data expand previous findings suggesting a complex role of the NK3-R in the acute behavioral effects of cocaine in non-human primates.