David Badre

Cognitive control, hierarchy, and the rostro-caudal organization of the frontal lobes

Cognitive control supports flexible behavior by selecting actions that are consistent with our goals and appropriate for our environment. The prefrontal cortex (PFC) has an established role in cognitive control, and research on the functional organization of PFC promises to contribute to our understanding of the architecture of control. A recently popular hypothesis is that the rostro-caudal axis of PFC supports a control hierarchy whereby posterior-to-anterior PFC mediates progressively abstract, higher-order control. This review discusses evidence for a rostro-caudal gradient of function in PFC and the theories proposed to account for these results, including domain generality in working memory, relational complexity, the temporal organization of behavior and abstract representational hierarchy. Distinctions among these frameworks are considered as a basis for future research.

Is the rostro-caudal axis of the frontal lobe hierarchical?

The frontal lobes in the brain are a component of the cerebral system that supports goal-directed behaviour. However, their functional organization remains controversial. Recent studies have reported rostro-caudal distinctions in frontal cortex activity based on the abstractness of action representations. In addition, some have proposed that these differences reflect a hierarchical organization, whereby anterior frontal regions influence processing by posterior frontal regions during the realization of abstract action goals as motor acts. However, few have considered whether the anatomy and physiology of the frontal lobes support such a scheme. To address this gap, this Review surveys anatomical, neuroimaging, electrophysiological and developmental findings, and considers the question: could the organization of the frontal cortex be hierarchical?

Functional magnetic resonance imaging evidence for a hierarchical organization of the prefrontal cortex

The prefrontal cortex (PFC) is central to flexible and organized action. Recent theoretical and empirical results suggest that the rostro-caudal axis of the frontal lobes may reflect a hierarchical organization of control. Here, we test whether the rostro-caudal axis of the PFC is organized hierarchically, based on the level of abstraction at which multiple representations compete to guide selection of action. Four functional magnetic resonance imaging (fMRI) experiments parametrically manipulated the set of task-relevant (a) responses, (b) features, (c) dimensions, and (d) overlapping cue-to-dimension mappings. A systematic posterior to anterior gradient was evident within the PFC depending on the manipulated level of representation. Furthermore, across four fMRI experiments, activation in PFC subregions was consistent with the sub- and superordinate relationships that define an abstract representational hierarchy. In addition to providing further support for a representational hierarchy account of the rostro-caudal gradient in the PFC, these data provide important empirical constraints on current theorizing about control hierarchies and the PFC.

Selection, integration, and conflict monitoring; assessing the nature and generality of prefrontal cognitive control mechanisms.

Prefrontal cortex (PFC) supports flexible behavior by mediating cognitive control, though the elemental forms of control supported by PFC remain a central debate. Dorsolateral PFC (DLPFC) is thought to guide response selection under conditions of response conflict or, alternatively, may refresh recently active representations within working memory. Lateral frontopolar cortex (FPC) may also adjudicate response conflict, though others propose that FPC supports higher order control processes such as subgoaling and integration. Anterior cingulate cortex (ACC) is hypothesized to upregulate response selection by detecting response conflict; it remains unclear whether ACC functions generalize beyond monitoring response conflict. The present fMRI experiment directly tested these competing theories regarding the functional roles of DLPFC, FPC, and ACC. Results reveal dissociable control processes in PFC, with mid-DLPFC selectively mediating resolution of response conflict and FPC further mediating subgoaling/integration. ACC demonstrated a broad sensitivity to control demands, suggesting a generalized role in modulating cognitive control.

Hierarchical cognitive control deficits following damage to the human frontal lobe

Cognitive control permits us to make decisions about abstract actions, such as whether to e-mail versus call a friend, and to select the concrete motor programs required to produce those actions, based on our goals and knowledge. The frontal lobes are necessary for cognitive control at all levels of abstraction. Recent neuroimaging data have motivated the hypothesis that the frontal lobes are organized hierarchically, such that control is supported in progressively caudal regions as decisions are made at more concrete levels of action. We found that frontal damage impaired action decisions at a level of abstraction that was dependent on lesion location (rostral lesions affected more abstract tasks, whereas caudal lesions affected more concrete tasks), in addition to impairing tasks requiring more, but not less, abstract action control. Moreover, two adjacent regions were distinguished on the basis of the level of control, consistent with previous functional magnetic resonance imaging results. These results provide direct evidence for a rostro-caudal hierarchical organization of the frontal lobes.

Semantic Retrieval, Mnemonic Control, and Prefrontal Cortex

Accessing stored knowledge is a fundamental function of the cognitive and neural architectures of memory. Here, the authors review evidence from cognitive-behavioral paradigms, neuropsychological studies of patients with focal neural insult, and functional brain imaging concerning the mechanisms underlying retrieval of semantic knowledge and their association with prefrontal cortex. First, the authors examine behavioral and neuropsychological evidence distinguishing between controlled and automatic semantic retrieval. Then the authors review the subregions of prefrontal cortex that functional neuroimaging has associated with semantic retrieval across a range of memory demanding tasks. Finally, two hypotheses concerning the nature of processing in these brain regions--the controlled semantic retrieval and selection hypotheses--are critically examined, and a possible synthesis is proposed.

Frontal cortex and the discovery of abstract action rules

Although we often encounter circumstances with which we have no prior experience, we rapidly learn how to behave in these novel situations. Such adaptive behavior relies on abstract behavioral rules that are generalizable, rather than concrete rules mapping specific cues to specific responses. Although the frontal cortex is known to support concrete rule learning, less well understood are the neural mechanisms supporting the acquisition of abstract rules. Here, we use a reinforcement learning paradigm to demonstrate that more anterior regions along the rostro-caudal axis of frontal cortex support rule learning at higher levels of abstraction. Moreover, these results indicate that when humans confront new rule learning problems, this rostro-caudal division of labor supports the search for relationships between context and action at multiple levels of abstraction simultaneously.

Mechanisms of Hierarchical Reinforcement Learning in Corticostriatal Circuits 1: Computational Analysis

Growing evidence suggests that the prefrontal cortex (PFC) is organized hierarchically, with more anterior regions having increasingly abstract representations. How does this organization support hierarchical cognitive control and the rapid discovery of abstract action rules? We present computational models at different levels of description. A neural circuit model simulates interacting corticostriatal circuits organized hierarchically. In each circuit, the basal ganglia gate frontal actions, with some striatal units gating the inputs to PFC and others gating the outputs to influence response selection. Learning at all of these levels is accomplished via dopaminergic reward prediction error signals in each corticostriatal circuit. This functionality allows the system to exhibit conditional if-then hypothesis testing and to learn rapidly in environments with hierarchical structure. We also develop a hybrid Bayesian-reinforcement learning mixture of experts (MoE) model, which can estimate the most likely hypothesis state of individual participants based on their observed sequence of choices and rewards. This model yields accurate probabilistic estimates about which hypotheses are attended by manipulating attentional states in the generative neural model and recovering them with the MoE model. This 2-pronged modeling approach leads to multiple quantitative predictions that are tested with functional magnetic resonance imaging in the companion paper.

Striatal contributions to declarative memory retrieval.

Declarative memory is known to depend on the medial temporal lobe memory system. Recently, there has been renewed focus on the relationship between the basal ganglia and declarative memory, including the involvement of striatum. However, the contribution of striatum to declarative memory retrieval remains unknown. Here, we review neuroimaging and neuropsychological evidence for the involvement of the striatum in declarative memory retrieval. From this review, we propose that, along with the prefrontal cortex (PFC), the striatum primarily supports cognitive control of memory retrieval. We conclude by proposing three hypotheses for the specific role of striatum in retrieval: (1) striatum modulates the re-encoding of retrieved items in accord with their expected utility (adaptive encoding), (2) striatum selectively admits information into working memory that is expected to increase the likelihood of successful retrieval (adaptive gating), and (3) striatum enacts adjustments in cognitive control based on the outcome of retrieval (reinforcement learning).

Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration.

How do individuals decide to act based on a rewarding status quo versus an unexplored choice that might yield a better outcome? Recent evidence suggests that individuals may strategically explore as a function of the relative uncertainty about the expected value of options. However, the neural mechanisms supporting uncertainty-driven exploration remain underspecified. The present fMRI study scanned a reinforcement learning task in which participants stop a rotating clock hand in order to win points. Reward schedules were such that expected value could increase, decrease, or remain constant with respect to time. We fit several mathematical models to subject behavior to generate trial-by-trial estimates of exploration as a function of relative uncertainty. These estimates were used to analyze our fMRI data. Results indicate that rostrolateral prefrontal cortex tracks trial-by-trial changes in relative uncertainty, and this pattern distinguished individuals who rely on relative uncertainty for their exploratory decisions versus those who do not.