Thomas A. Stalnaker

A new perspective on the role of the orbitofrontal cortex in adaptive behaviour

The orbitofrontal cortex (OFC) is crucial for changing established behaviour in the face of unexpected outcomes. This function has been attributed to the role of the OFC in response inhibition or to the idea that the OFC is a rapidly flexible associative-learning area. However, recent data contradict these accounts, and instead suggest that the OFC is crucial for signalling outcome expectancies. We suggest that this function — signalling of expected outcomes — can also explain the crucial role of the OFC in changing behaviour in the face of unexpected outcomes.

Orbitofrontal cortex, decision-making and drug addiction

The orbitofrontal cortex, as a part of prefrontal cortex, is implicated in executive function. However, within this broad region, the orbitofrontal cortex is distinguished by its unique pattern of connections with crucial subcortical associative learning nodes, such as basolateral amygdala and nucleus accumbens. By virtue of these connections, the orbitofrontal cortex is uniquely positioned to use associative information to project into the future, and to use the value of perceived or expected outcomes to guide decisions. This review will discuss recent evidence that supports this proposal and will examine evidence that loss of this signal, as the result of drug-induced changes in these brain circuits, might account for the maladaptive decision-making that characterizes drug addiction.

The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes

Humans and other animals change their behavior in response to unexpected outcomes. The orbitofrontal cortex (OFC) is implicated in such adaptive responding, based on evidence from reversal tasks. Yet these tasks confound using information about expected outcomes with learning when those expectations are violated. OFC is critical for the former function; here we show it is also critical for the latter. In a Pavlovian overexpectation task, inactivation of OFC prevented learning driven by unexpected outcomes, even when performance was assessed later. We propose this reflects a critical contribution of outcome signaling by OFC to encoding of reward prediction errors elsewhere. In accord with this proposal, we report that signaling of reward predictions by OFC neurons was related to signaling of prediction errors by dopamine neurons in ventral tegmental area (VTA). Furthermore, bilateral inactivation of VTA or contralateral inactivation of VTA and OFC disrupted learning driven by unexpected outcomes.

Basolateral Amygdala Lesions Abolish Orbitofrontal-Dependent Reversal Impairments

Damage to orbitofrontal cortex (OFC) has long been associated with deficits in reversal learning. OFC damage also causes inflexible associative encoding in basolateral amygdala (ABL) during reversal learning. Here we provide a critical test of the hypothesis that the reversal deficit in OFC-lesioned rats is caused by this inflexible encoding in ABL. Rats with bilateral neurotoxic lesions of OFC, ABL, or both areas were tested on a series of two-odor go/no-go discrimination problems, followed by two serial reversals of the final problem. As expected, all groups acquired the initial problems at the same rate, and rats with OFC lesions were slower to acquire the reversals than sham controls. This impairment was abolished by accompanying ABL lesions, while ABL lesions alone had no effect on reversal learning. These results are consistent with the hypothesis that OFC facilitates cognitive flexibility by promoting updating of associative encoding in downstream brain areas.

What the orbitofrontal cortex does not do

The number of papers about the orbitofrontal cortex (OFC) has grown from 1 per month in 1987 to a current rate of over 50 per month. This publication stream has implicated the OFC in nearly every function known to cognitive neuroscience and in most neuropsychiatric diseases. However, new ideas about OFC function are typically based on limited data sets and often ignore or minimize competing ideas or contradictory findings. Yet true progress in our understanding of an areas function comes as much from invalidating existing ideas as proposing new ones. Here we consider the proposed roles for OFC, critically examining the level of support for these claims and highlighting the data that call them into question.

Reconciling the Roles of Orbitofrontal Cortex in Reversal Learning and the Encoding of Outcome Expectancies

Abstract: Damage to orbitofrontal cortex (OFC) has long been associated with decision‐making deficits. Such deficits are epitomized by impairments in reversal learning. Historically, reversal learning deficits have been linked to a response inhibition function or to the rapid reversal of associative encoding in OFC neurons. However here we will suggest that OFC supports reversal learning not because its encoding is particularly flexible—indeed it actually is not—but rather because output from OFC is critical for flexible associative encoding downstream in basolateral amygdala (ABL). Consistent with this argument, we will show that reversal performance is actually inversely related to the flexibility of associative encoding in OFC (i.e., the better the reversal performance, the less flexible the encoding). Further, we will demonstrate that associative correlates in ABL are more flexible during reversal learning than in OFC, become less flexible after damage to OFC, and are required for the expression of the reversal deficit caused by OFC lesions. We will propose that OFC facilitates associative flexibility in downstream regions, such as ABL, for the same reason that it is critical for outcome‐guided behavior in a variety of setting—namely that processing in OFC signals the value of expected outcomes. In addition to their role in guiding behavior, these outcome expectancies permit the rapid recognition of unexpected outcomes, thereby driving new learning.

Neural substrates of cognitive inflexibility after chronic cocaine exposure

Cognitive changes in addicts and animals exposed to addictive drugs have been extensively investigated over the past decades. One advantage of studying addiction using cognitive paradigms is that neural processing in addicts or drug-exposed animals can be compared to that in normal subjects. Tests of cognitive flexibility that measure the ability to change responding to a previously rewarded or punished stimulus are of potential interest in the study of addiction, because addiction can itself be viewed as an inability to change responding to stimuli previously associated with drug reward. One such test is reversal learning, which is impaired in cocaine addicts and animals that have chronically self-administered or been exposed to cocaine. A circuit including orbitofrontal cortex, basolateral amygdala and striatum subserves reversal learning. In rats that have been previously exposed to cocaine, neurons in these regions show selective and distinct changes in how they encode information during reversal learning. These changes suggest that in these rats, orbitofrontal cortex loses the ability to signal expected outcomes, and basolateral amygdala becomes inflexible in its encoding of cue significance. These changes could explain cocaine-induced impairments to cognitive flexibility and may have theoretical importance in addiction.

Abnormal associative encoding in orbitofrontal neurons in cocaine-experienced rats during decision-making

Recent evidence has linked exposure to addictive drugs to an inability to employ information about adverse consequences, or outcomes, to control behavior. For instance, addicts and drug‐experienced animals fail to adapt their behavior to avoid adverse outcomes in gambling and reversal tasks or after changes in the value of expected rewards. These deficits are similar to those caused by damage to the orbitofrontal cortex, suggesting that addictive drugs may cause long‐lasting changes in the representation of outcome associations in a circuit that includes the orbitofrontal cortex. Here we test this hypothesis by recording from orbitofrontal neurons in a discrimination task in rats previously exposed to cocaine (30 mg/kg i.p. for 14 days). We found that orbitofrontal neurons recorded in cocaine‐experienced rats failed to signal the adverse outcome at the time a decision was made in the task. The loss of this signal was associated with abnormal changes in response latencies on aversive trials. Furthermore, upon reversal of the cue–outcome associations, orbitofrontal neurons in cocaine‐treated rats with enduring reversal impairments failed to reverse their cue‐selectivity, while orbitofrontal neurons in cocaine‐treated rats with normal performance showed an increase in the plasticity of cue‐selective firing after reversal. These results provide direct neurophysiological evidence that exposure to cocaine can cause behaviorally relevant changes in the processing of associative information in a circuit that includes the orbitofrontal cortex.

Neural correlates of stimulus-response and response-outcome associations in dorsolateral versus dorsomedial striatum

Considerable evidence suggests that there is functional heterogeneity in the control of behavior by the dorsal striatum. Dorsomedial striatum may support goal-directed behavior by representing associations between responses and outcomes (R–O associations). The dorsolateral striatum, in contrast, may support motor habits by encoding associations between stimuli and responses (S–R associations). To test whether neural correlates in striatum in fact conform to this pattern, we recorded single-units in dorsomedial and dorsolateral striatum of rats performing a task in which R–O contingencies were manipulated independently of S–R contingencies. Among response-selective neurons in both regions, activity was significantly modulated by the initial stimulus, providing evidence of S–R encoding. Similarly, response selectivity was significantly modulated by the associated outcome in both regions, providing evidence of R–O encoding. In both regions, this outcome-modulation did not seem to reflect the relative value of the expected outcome, but rather its specific identity. Finally, in both regions we found correlates of the available action–outcome contingencies reflected in the baseline activity of many neurons. These results suggest that differences in information content in these two regions may not determine the differential roles they play in controlling behavior, demonstrated in previous studies.

Cocaine exposure shifts the balance of associative encoding from ventral to dorsolateral striatum.

Both dorsal and ventral striatum are implicated in the “habitization” of behavior that occurs in addiction. Here we examined the effect of cocaine exposure on associative encoding in these two regions. Neural activity was recorded during go/no-go discrimination learning and reversal. Activity in ventral striatum developed and reversed rapidly, tracking the valence of the predicted outcome, whereas activity in dorsolateral striatum developed and reversed more slowly, tracking discriminative responding. This difference is consistent with the putative roles of these two areas in promoting habit-like behavior. Dorsolateral striatum has been directly implicated in habit or stimulus–response learning, whereas ventral striatum appears to be involved indirectly by allowing cues associated with reward to exert a general motivational influence on responding. Interestingly cocaine exposure did not uniformly enhance processing across both regions. Instead cocaine reduced the degree and flexibility of cue-evoked firing in ventral striatum while marginally enhanced cue-selective firing in dorsolateral striatum. Thus cocaine exposure causes regionally specific effects on neural processing in striatum; these effects may promote the habitization of behavior by shifting control from ventral to dorsolateral regions.