Andrew Gillies

Subthalamic-pallidal interactions are critical in determining normal and abnormal functioning of the basal ganglia.

The subthalamic nucleus (STN) and external globus pallidus (GP) form a recurrent excitatory–inhibitory interaction within the basal ganglia. Through a computational model of these interactions we show that, under the influence of appropriate external input, the two nuclei can be switched between states of high and low activity or can generate oscillations consisting of bursts of high-frequency activity repeated at a low rate. It is further demonstrated from the model that the generation of the repetitive bursting behaviour is favoured by increased inhibition of the GP, which is a condition indicated in Parkinsons disease. Paradoxically, increased striatal inhibition of the GP is predicted to cause an increase rather than a decrease in its mean firing rate. These behaviours, arising from a biologically inspired computational model of the STN–GP interaction, have important consequences for basal ganglia function and dysfunction.

Computational models of the basal ganglia

Computer simulation studies and mathematical analysis of models of the basal ganglia are being used increasingly to explore theories of basal ganglia function. We review the implications of these new models for a general understanding of basal ganglia function in normal as well as in diseased brains. The focus is on their functional similarities rather than on the details of mathematical methodologies and simulation techniques. Most of the models suggest a vital role for the basal ganglia in learning. Although this interest in learning is partly driven by experimental results associating the acute firing of dopamine cells with reward prediction in monkeys, some of the models have preceded the electrophysiological results. An‐ other common theme of the models is selection. In this case, the striatum is seen as detecting and selecting cortical contexts for access to basal ganglia output. Although the behavioral consequences of this selection are hard to define, the models provide frameworks within which to explore these ideas empirically. This provides a means of refining our understanding of basal ganglia function and to consider dysfunction within the new logical frameworks.

Hedonic value: enhancing adaptation for motivated agents

Reinforcement learning (RL) in the context of artificial agents is typically used to produce behavioral responses as a function of the reward obtained by interaction with the environment. When the problem consists of learning the shortest path to a goal, it is common to use reward functions yielding a fixed value after each decision, for example a positive value if the target location has been attained and a negative value at each intermediate step. However, this fixed strategy may be overly simplistic for agents to adapt to dynamic environments, in which resources may vary from time to time. By contrast, there is significant evidence that most living beings internally modulate reward value as a function of their context to expand their range of adaptivity. Inspired by the potential of this operation, we present a review of its underlying processes and we introduce a simplified formalization for artificial agents. The performance of this formalism is tested by monitoring the adaptation of an agent endowed with a model of motivated actor–critic, embedded with our formalization of value and constrained by physiological stability, to environments with different resource distribution. Our main result shows that the manner in which reward is internally processed as a function of the agent’s motivational state, strongly influences adaptivity of the behavioral cycles generated and the agent’s physiological stability.

Neuroinformatics and modeling of the basal ganglia: bridging pharmacology and physiology

The subthalamic nucleus (STN) is the primary target for the chronic deep brain stimulation treatment of Parkinson’s disease. STN neurons exhibit a variety of characteristic properties that may play a key role in the overall population response to deep brain stimulation. Neuroinformatics techniques, in particular computational modeling, provide a method of bringing together pharmacological phenomena, such as the loss of dopamine, with electrophysiological characteristics. Developing accurate models of STN neurons plays an important part in the process of uncovering the link between the changes in STN pharmacology, physiology and synaptic input that occurs with Parkinson’s disease and the effectiveness of treatments targeting the STN. We review a general procedure for developing computational models and present a model of STN neurons that reveals important membrane channel interactions. In particular, changes in these channel interactions under parkinsonian conditions may underlie changes in characteristic physiology, critical in determining the mechanisms of deep brain stimulation.

Functional Interactions within the Subthalamic Nucleus

The subthalamic nucleus (STN) is at the centre of basal ganglia processing. It has excitatory projections to both primary output nuclei of the basal ganglia (globus pallidus internal segment (GPi) and substantia nigra pars reticulata (SNr)) and a major projection to the external pallidal segment (GPe) (Carpenter, 1981; Smithet al.1994). It receives recurrent feedback from the GPe (Joel & Weiner, 1997; Shinket al.1996; Smithet al.1994), direct cortical and thalamic input (Parent, 1990; Parent & Hazrati, 1993; Sugimotoet al.1985) and dopaminergic innervation from the substantia nigra pars compacta (SNc) (Hassaniet al.1997). The STN is a key target in current surgical treatments of Parkinson’s disease (Limousinet al.1995; Bergmanet al.1990). Its operation is fundamental to the operation of both the indirect network through the basal ganglia and to basal ganglia processing as a whole (Ryan & Sanders, 1993). However, there is limited information on how the subthalamic nucleus functions and processes information. Most proposals originate from high level models which simply require the nucleus to operate in a certain way (for a review of high level computational models of the basal ganglia, see Gillies & Arbuthnott, 2000). We present a synthesis of multilevel computational and mathematical models each based on properties of the STN and its interactions in order to expose its primary modes of operation. We bring together models of neuroanatomy, dynamic network models and a detailed model of a recent pharmacological experiment to frame the modes of STN operation and highlight important properties underlying these.