Steven W. Kennerley

Optimal decision making and the anterior cingulate cortex

Learning the value of options in an uncertain environment is central to optimal decision making. The anterior cingulate cortex (ACC) has been implicated in using reinforcement information to control behavior. Here we demonstrate that the ACCs critical role in reinforcement-guided behavior is neither in detecting nor in correcting errors, but in guiding voluntary choices based on the history of actions and outcomes. ACC lesions did not impair the performance of monkeys (Macaca mulatta) immediately after errors, but made them unable to sustain rewarded responses in a reinforcement-guided choice task and to integrate risk and payoff in a dynamic foraging task. These data suggest that the ACC is essential for learning the value of actions.

Neurons in the frontal lobe encode the value of multiple decision variables

A central question in behavioral science is how we select among choice alternatives to obtain consistently the most beneficial outcomes. Three variables are particularly important when making a decision: the potential payoff, the probability of success, and the cost in terms of time and effort. A key brain region in decision making is the frontal cortex as damage here impairs the ability to make optimal choices across a range of decision types. We simultaneously recorded the activity of multiple single neurons in the frontal cortex while subjects made choices involving the three aforementioned decision variables. This enabled us to contrast the relative contribution of the anterior cingulate cortex (ACC), the orbito-frontal cortex, and the lateral prefrontal cortex to the decision-making process. Neurons in all three areas encoded value relating to choices involving probability, payoff, or cost manipulations. However, the most significant signals were in the ACC, where neurons encoded multiplexed representations of the three different decision variables. This supports the notion that the ACC is an important component of the neural circuitry underlying optimal decision making.

Double dissociation of value computations in orbitofrontal and anterior cingulate neurons

Damage to prefrontal cortex (PFC) impairs decision-making, but the underlying value computations that might cause such impairments remain unclear. Here we report that value computations are doubly dissociable among PFC neurons. Although many PFC neurons encoded chosen value, they used opponent encoding schemes such that averaging the neuronal population extinguished value coding. However, a special population of neurons in anterior cingulate cortex (ACC), but not in orbitofrontal cortex (OFC), multiplexed chosen value across decision parameters using a unified encoding scheme and encoded reward prediction errors. In contrast, neurons in OFC, but not ACC, encoded chosen value relative to the recent history of choice values. Together, these results suggest complementary valuation processes across PFC areas: OFC neurons dynamically evaluate current choices relative to recent choice values, whereas ACC neurons encode choice predictions and prediction errors using a common valuation currency reflecting the integration of multiple decision parameters.

Frontal Cortex Subregions Play Distinct Roles in Choices between Actions and Stimuli

The orbitofrontal cortex (OFC) has been implicated in reinforcement-guided decision making, error monitoring, and the reversal of behavior in response to changing circumstances. The anterior cingulate cortex sulcus (ACCS), however, has also been implicated in similar aspects of behavior. Dissociating the unique functions of these areas would improve our understanding of the decision-making process. The effect of selective OFC lesions on how monkeys used the history of reinforcement to guide choices of either particular actions or particular stimuli was studied and compared with the effects of ACCS lesions. Both lesions disrupted decision making, but their effects were differentially modulated by the dependence on action– or stimulus–value contingencies. OFC lesions caused a deficit in stimulus but not action selection, whereas ACCS lesions had the opposite effect, disrupting action but not stimulus selection. Furthermore, OFC lesions that have previously been found to impair decision making when deterministic stimulus–reward contingencies are switched were found to cause a more general learning impairment in more naturalistic situations in which reward was stochastic. Both OFC and ACCS are essential for reinforcement-guided decision making rather than just error monitoring or behavioral reversal. The OFC and ACCS are both, however, more concerned with learning and making decisions, but their roles in selecting between stimulus and action values are distinct.

Callosotomy patients exhibit temporal uncoupling during continuous bimanual movements

Rhythmic bimanual movements are highly constrained in the temporal domain, with the gestures of the two hands tightly synchronized. Previous studies have implicated a subcortical locus for temporal coupling based on the observation that these constraints persist in callosotomy patients. We now report that such coupling is restricted to movements entailing a discrete event (such as a movement onset). Three callosotomy patients exhibited a striking lack of temporal coupling during continuous movements, with the two hands oscillating at non-identical frequencies. We propose a subcortical locus of temporal coupling for movements involving discrete events. In contrast, synchronization between the hands during continuous movements depends on interhemispheric transmission across the corpus callosum.

Oscillatory phase coupling coordinates anatomically dispersed functional cell assemblies

Hebb proposed that neuronal cell assemblies are critical for effective perception, cognition, and action. However, evidence for brain mechanisms that coordinate multiple coactive assemblies remains lacking. Neuronal oscillations have been suggested as one possible mechanism for cell assembly coordination. Prior studies have shown that spike timing depends upon local field potential (LFP) phase proximal to the cell body, but few studies have examined the dependence of spiking on distal LFP phases in other brain areas far from the neuron or the influence of LFP–LFP phase coupling between distal areas on spiking. We investigated these interactions by recording LFPs and single-unit activity using multiple microelectrode arrays in several brain areas and then used a unique probabilistic multivariate phase distribution to model the dependence of spike timing on the full pattern of proximal LFP phases, distal LFP phases, and LFP–LFP phase coupling between electrodes. Here we show that spiking activity in single neurons and neuronal ensembles depends on dynamic patterns of oscillatory phase coupling between multiple brain areas, in addition to the effects of proximal LFP phase. Neurons that prefer similar patterns of phase coupling exhibit similar changes in spike rates, whereas neurons with different preferences show divergent responses, providing a basic mechanism to bind different neurons together into coordinated cell assemblies. Surprisingly, phase-coupling–based rate correlations are independent of interneuron distance. Phase-coupling preferences correlate with behavior and neural function and remain stable over multiple days. These findings suggest that neuronal oscillations enable selective and dynamic control of distributed functional cell assemblies.

Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables.

Damage to the frontal lobe can cause severe decision‐making impairments. A mechanism that may underlie this is that neurons in the frontal cortex encode many variables that contribute to the valuation of a choice, such as its costs, benefits and probability of success. However, optimal decision‐making requires that one considers these variables, not only when faced with the choice, but also when evaluating the outcome of the choice, in order to adapt future behaviour appropriately. To examine the role of the frontal cortex in encoding the value of different choice outcomes, we simultaneously recorded the activity of multiple single neurons in the anterior cingulate cortex (ACC), orbitofrontal cortex (OFC) and lateral prefrontal cortex (LPFC) while subjects evaluated the outcome of choices involving manipulations of probability, payoff and cost. Frontal neurons encoded many of the parameters that enabled the calculation of the value of these variables, including the onset and offset of reward and the amount of work performed, and often encoded the value of outcomes across multiple decision variables. In addition, many neurons encoded both the predicted outcome during the choice phase of the task as well as the experienced outcome in the outcome phase of the task. These patterns of selectivity were more prevalent in ACC relative to OFC and LPFC. These results support a role for the frontal cortex, principally ACC, in selecting between choice alternatives and evaluating the outcome of that selection thereby ensuring that choices are optimal and adaptive.

Heterogeneous reward signals in prefrontal cortex

Neurons encode upcoming rewards throughout frontal cortex. Recent papers have helped to determine that these signals play different roles in different frontal regions. Neurons in orbitofrontal cortex (PFo) appear to be responsible for calculating the specific value of an expected reward, information that can help efficiently guide decision-making. Similar signals are also present in the cingulate sulcus (PFcs). By contrast, reward signals in lateral prefrontal cortex (PFl) are consistent with a role in using reward to guide other cognitive processes, such as the allocation of attentional resources and using value information to guide learning other relationships in the environment such as arbitrary stimulus-response mappings. A remaining issue for future work is to specify the precise roles of PFo and PFcs. These two areas show very different patterns of connectivity with other brain areas, and it is currently unclear how this effects their contribution to decision-making.

Moving to Directly Cued Locations Abolishes Spatial Interference During Bimanual Actions

Interference is frequently observed during bimanual movements if the two hands perform nonsymmetric actions. We examined the source of bimanual interference in two experiments in which we compared conditions involving symmetric movements with conditions in which the movements were of different amplitudes or different directions. The target movements were cued either symbolically by letters or directly by the onset of the target locations. With symbolic cues, reaction times were longer when the movements of the two hands were not symmetric. With direct cues, reaction times were the same for symmetric and nonsymmetric movements. These results indicate that directly cued actions can be programmed in parallel for the two hands. Our results challenge the hypothesis that the cost to initiate nonsymmetric movements is due to spatial interference in a motor-programming stage. Rather, the cost appears to be caused by stimulus identification, response-selection processes connected to the processing of symbolic cues, or both.

Reward-dependent modulation of working memory in lateral prefrontal cortex

Although research implicates lateral prefrontal cortex (PFC) in executive control and goal-directed behavior, it remains unclear how goals influence executive processes. One possibility is that goal-relevant information, such as expected rewards, could modulate the representation of information relating to executive control, thereby ensuring the efficient allocation of cognitive resources. To investigate this, we examined how reward modulated spatial working memory. Past studies investigating spatial working memory have focused on dorsolateral PFC, but this area only weakly connects with areas processing reward. Ventrolateral PFC has better connections in this regard. Thus, we contrasted the functional properties of single neurons in ventrolateral and dorsolateral PFC as two subjects performed a task that required them to hold spatial information in working memory under different expectancies of reward for correct performance. We balanced the order of presentation of spatial and reward information so we could assess the neuronal encoding of the two pieces of information independently and conjointly. Neurons in ventrolateral PFC encoded both spatial and reward information earlier, stronger and in a more sustained manner than neurons in dorsolateral PFC. Within ventrolateral PFC, spatial selectivity was more prevalent on the inferior convexity than within the principal sulcus. Finally, when reward increased spatial selectivity, behavioral performance improved, whereas when reward decreased spatial selectivity, behavioral performance deteriorated. These results suggest that ventrolateral PFC may be a locus whereby information about expected rewards can modulate information in working memory. The pattern of results is consistent with a role for ventrolateral PFC in attentional control.