Meropi Topalidou

The globus pallidus pars interna in goal-oriented and routine behaviors: Resolving a long-standing paradox

There is an apparent contradiction between experimental data showing that the basal ganglia are involved in goal‐oriented and routine behaviors and clinical observations. Lesion or disruption by deep brain stimulation of the globus pallidus interna has been used for various therapeutic purposes ranging from the improvement of dystonia to the treatment of Tourettes syndrome. None of these approaches has reported any severe impairment in goal‐oriented or automatic movement.

One critic, two actors: evidence for covert learning in the basal ganglia

This paper introduces a new hypothesis concerning the dissociated role of the basal ganglia in the selection and the evaluation of action that has been formulated using a theoretical model and confirmed experimentally in monkeys. To do so, prior to learning, we inactivated the internal part of the Globus Pallidus (GPi, the main output structure of the BG) with injections of muscimol and we tested monkeys on a variant of a two-armed bandit task where two stimuli are associated with two distinct reward probabilities (0.25 and 0.75 respectively). Unsurprisingly, performance in such condition are at the chance level because the output of basal ganglia is suppressed and they cannot influence behaviour. However, the theoretical model predicts that in the meantime, values of the stimuli are nonetheless covertly evaluated and learned. This has been tested and confirmed on the next day, when muscimol has been replaced by a saline solution: monkeys instantly showed significantly improved performances (above chance level), hence demonstrating they have covertly learned the relative value of the two stimuli. This tends to suggest a competition takes place in the Cortex-BG loop between two actors, one of whom being sensitive to criticism and the other not. Ultimately, the actual choice is valuated, independently of the origin of the decision.This paper introduces a new hypothesis concerning the formation of habits in the cortex of primates under the implicit supervision of the basal ganglia. This hypothesis has been formulated using a theoretical model and confirmed experimentally in monkeys. To do so, and prior to learning, we inactivated the internal part of the globus pallidus (GPi, the main output structure of the BG) with injections of muscimol and we tested monkeys on a variant of a two-armed bandit task where two stimuli are associated with two distinct reward probabilities (0.25 and 0.75 respectively). Unsurprisingly, their performance in such conditions are at the chance level. However, the theoretical model predicts that even if the performance is random, the value of the stimuli are implicitly evaluated and learned. This has been tested and confirmed on the next day, when inhibition has been removed: monkeys instantly showed quasi-optimal performances, demonstrating they knew the relative value of the two stimuli. Said differently, we managed to explicitly dissociate reinforcement learning from Hebbian learning and demonstrated covert learning inside the basal ganglia. These results suggest that a behavioral decision results from both the cooperation (acquisition) and competition (expression) of two distinct but entangled memory systems, the goal-directed system and the habit system that may represent the two ends of the same graded phenomenon.

Dual Competition between the Basal Ganglia and the Cortex: from Action-Outcome to Stimulus-Response

Action-outcome (A-O) and stimulus-response (S-R) processes that are two forms of instrumental conditioning that are important components of decision making and action selection. The former adapts its response according to the outcome while the latter is insensitive to the outcome. An unsolved question is how these two processes emerge, cooperate and interact inside the brain in order to issue a unique behavioral answer. Here we propose a model of the interaction between the cortex, the basal ganglia and the thalamus based on a dual competition. We hypothesize that the striatum, the subthalamic nucleus, the internal pallidum (GPi), the thalamus, and the cortex are involved in closed feedback loops through the hyperdirect and direct pathways. These loops support a competition process that results in the ability for the basal ganglia to make a cognitive decision followed by a motor decision. Considering lateral cortical interactions (short range excitation, long range inhibition), another competition takes place inside the cortex allowing this latter to make a cognitive and a motor decision. We show how this dual competition endows the model with two regimes. One is oriented towards action-outcome and is driven by reinforcement learning, the other is oriented towards stimulus-response and is driven by Hebbian learning. The final decision is made according to a combination of these two mechanisms with a gradual transfer from the former to the latter. We confirmed these theoretical results on primates using a two-armed bandit task and a reversible bilateral inactivation of the internal part of the globus pallidus.

The formation of habits in the neocortex under the implicit supervision of the basal ganglia

If basal ganglia are widely accepted to participate in the high-level cognitive function of decision-making, their role is less clear regarding the formation of habits [1,2]. One of the biggest problem is to understand how goal-directed actions are transformed into habitual responses, or, said differently, how an animal can shift from an action-outcome (A-O) system to a stimulus-response (S-R) one, while maintaining a consistent behavior. We introduce a computational model (basal ganglia, thalamus and cortex) that can solve a simple two arm-bandit task using reinforcement learning and explicit valuation of the outcome [3]. Hebbian learning has been implemented at the cortical level such that the model learns each time a move is issued, rewarded or not. Then, by inhibiting the output nuclei of the model (GPi), we show how learning has been transferred from the basal ganglia to the cortex, simply as a consequence of the statistics of the different choices (see Figure ​Figure1).1). Because best (in the sense of most rewarded) actions are chosen more often, this directly impacts the amount of Hebbian learning and lead to the formation of habits within the cortex. These results have been confirmed in monkeys doing the exact same task where the BG has been inactivated using muscimol. This tends to show that the basal ganglia implicitly teach the cortex in order for it to learn the values of new options. In the end, the cortex is able to solve the task perfectly, even if it exhibits slower reaction times. Figure 1 Left. In habitual condition (HC), performances are optimal, with or without GPi. In novel condition (NC), only the intact model (with GPi) is able to learn the new stimuli while lesioned model performances stay at the level f chance. Right. Analysis of ...