Genela Morris

Midbrain dopamine neurons encode decisions for future action

Current models of the basal ganglia and dopamine neurons emphasize their role in reinforcement learning. However, the role of dopamine neurons in decision making is still unclear. We recorded from dopamine neurons in monkeys engaged in two types of trial: reference trials in an instructed-choice task and decision trials in a two-armed bandit decision task. We show that the activity of dopamine neurons in the decision setting is modulated according to the value of the upcoming action. Moreover, analysis of the probability matching strategy in the decision trials revealed that the dopamine population activity and not the reward during reference trials determines choice behavior. Because dopamine neurons do not have spatial or motor properties, we conclude that immediate decisions are likely to be generated elsewhere and conveyed to the dopamine neurons, which play a role in shaping long-term decision policy through dynamic modulation of the efficacy of basal ganglia synapses.

Independent Coding of Movement Direction and Reward Prediction by Single Pallidal Neurons

Associating action with its reward value is a basic ability needed by adaptive organisms and requires the convergence of limbic, motor, and associative information. To chart the basal ganglia (BG) involvement in this association, we recorded the activity of 61 well isolated neurons in the external segment of the globus pallidus (GPe) of two monkeys performing a probabilistic visuomotor task. Our results indicate that most (96%) neurons responded to multiple phases of the task. The activity of many (34%) pallidal neurons was modulated solely by direction of movement, and the activity of only a few (3%) pallidal neurons was modulated exclusively by reward prediction. However, the activity of a large number (41%) of single pallidal neurons was comodulated by both expected trial outcome and direction of arm movement. The information carried by the neuronal activity of single pallidal neurons dynamically changed as the trial progressed. The activity was predominantly modulated by both outcome prediction and future movement direction at the beginning of trials and became modulated mainly by movement-direction toward the end of trials. GPe neurons can either increase or decrease their discharge rate in response to predicted future reward. The effects of movement-direction and reward probability on neural activity are linearly summed and thus reflect two independent modulations of pallidal activity. We propose that GPe neurons are uniquely suited for independent processing of a multitude of parameters. This is enabled by the funnel-structure characteristic of the BG architecture, as well as by the anatomical and physiological properties of GPe neurons.

Statistical Properties of Pauses of the High-Frequency Discharge Neurons in the External Segment of the Globus Pallidus

The neurons of many basal ganglia nuclei, including the external and internal globus pallidus (GPe and GPi, respectively) and the substantia nigra pars reticulata (SNr) are characterized by their high-frequency (50–100 spikes/s) tonic discharge (HFD). However, the high firing rate of GPe neurons is interrupted by long pauses. We studied the extracellularly recorded spiking activity of 212 well-isolated HFD GPe and 52 GPi/SNr neurons from five monkeys during different states of behavioral activity. An algorithm that maximizes the surprise function was used to detect pauses and pauser cells (“pausers”). Only 6% of the GPi/SNr neurons versus as many as 56% of the GPe neurons were classified as pausers. The GPe average pause duration equals 0.62 s. The interpause intervals follow a Poissonian distribution with a frequency of 13 pauses/minute. No linear relationship was found between pause parameters (duration or frequency) and the firing rate of the cell. Pauses were preceded by various changes in firing rate but not dominantly by a decrease. The average amplitude and duration of the spike waveform was modulated only after the pause but not before it. Pauses of pairs of cells that were recorded simultaneously were not correlated. The probability of GPe cells to pause spontaneously was extremely variable among monkeys (30–90%) and inversely related to the degree of the monkeys motor activity. These findings suggest that spontaneous GPe pauses are related to low-arousal periods and are generated by a process that is independent of the discharge properties of the cells.

ECOLOGY AND NEUROBIOLOGY OF TOXIN AVOIDANCE AND THE PARADOX OF DRUG REWARD

Current neurobiological theory of drug use is based on the observation that all addictive drugs induce changes in activity of dopaminergic circuitry, interfering with reward processing, and thus enhancing drug seeking and consumption behaviors. Current theory of drug origins, in contrast, views almost all major drugs of abuse, including nicotine, cocaine and opiates, as plant neurotoxins that evolved to punish and deter herbivores. According to this latter view, plants should not have evolved compounds that reward or reinforce plant consumption. Mammals, in turn, should not have evolved reinforcement mechanisms easily triggered by toxic substances. Situated in an ecological context, therefore, drug reward is a paradox. In an attempt to resolve the paradox, we review the neurobiology of aversive learning and toxin avoidance and their relationships to appetitive learning. We seek to answer the question: why does aversive learning not prevent the repeated use of plant drugs? We conclude by proposing alternative models of drug seeking and use. Specifically, we suggest that humans, like other animals, might have evolved to counter-exploit plant neurotoxins.

An Approach for Reliably Investigating Hippocampal Sharp Wave-Ripples In Vitro

Background Among the various hippocampal network patterns, sharp wave-ripples (SPW-R) are currently the mechanistically least understood. Although accurate information on synaptic interactions between the participating neurons is essential for comprehensive understanding of the network function during complex activities like SPW-R, such knowledge is currently notably scarce. Methodology/Principal Findings We demonstrate an in vitro approach to SPW-R that offers a simple experimental tool allowing detailed analysis of mechanisms governing the sharp wave-state of the hippocampus. We combine interface storage of slices with modifications of a conventional submerged recording system and established in vitro SPW-R comparable to their in vivo counterpart. We show that slice storage in the interface chamber close to physiological temperature is the required condition to preserve network integrity that is necessary for the generation of SPW-R. Moreover, we demonstrate the utility of our method for studying synaptic and network properties of SPW-R, using electrophysiological and imaging methods that can only be applied in the submerged system. Conclusions/Significance The approach presented here demonstrates a reliable and experimentally simple strategy for studying hippocampal sharp wave-ripples. Given its utility and easy application we expect our model to foster the generation of new insight into the network physiology underlying SPW-R.

Striatal action-learning based on dopamine concentration

The reinforcement learning hypothesis of dopamine function predicts that dopamine acts as a teaching signal by governing synaptic plasticity in the striatum. Induced changes in synaptic strength enable the cortico-striatal network to learn a mapping between situations and actions that lead to a reward. A review of the relevant neurophysiology of dopamine function in the cortico-striatal network and the machine reinforcement learning hypothesis reveals an apparent mismatch with recent electrophysiological studies. It was found that in addition to the well-described reward-related responses, a subpopulation of dopamine neurons also exhibits phasic responses to aversive stimuli or to cues predicting aversive stimuli. Obviously, actions that lead to aversive events should not be reinforced. However, published data suggest that the phasic responses of dopamine neurons to reward-related stimuli have a higher firing rate and have a longer duration than phasic responses of dopamine neurons to aversion-related stimuli. We propose that based on different dopamine concentrations, the target structures are able to decode reward-related dopamine from aversion-related dopamine responses. Thereby, the learning of actions in the basal-ganglia network integrates information about both costs and benefits. This hypothesis predicts that dopamine concentration should be a crucial parameter for plasticity rules at cortico-striatal synapses. Recent in vitro studies on cortico-striatal synaptic plasticity rules support a striatal action-learning scheme where during reward-related dopamine release dopamine-dependent forms of synaptic plasticity occur, while during aversion-related dopamine release the dopamine concentration only allows dopamine-independent forms of synaptic plasticity to occur.

Cannabinoids disrupt hippocampal sharp wave-ripples via inhibition of glutamate release

Cannabis consumption results in impaired learning. The proper synchronization of neuronal activity in the mammalian hippocampus gives rise to network rhythms that are implicated in memory formation. Here, we have studied the impact of cannabinoids on hippocampal sharp waves and associated ripple oscillations using field‐ and whole‐cell voltage‐clamp recordings. We demonstrate that the activation of cannabinoid receptor 1 suppresses sharp wave‐ripples (SWRs) in mice in vivo and in vitro. This suppression was paralleled by a selective reduction of SWR‐associated inward but not outward charge transfer, demonstrating an impairment of excitation due to cannabinoid exposure. Adenosine, a presynaptic modulator of glutamate release, mimicked and occluded the observed consequences of cannabinoids on SWRs. We conclude that inhibition of glutamatergic feed‐forward excitation can explain cannabinoid‐mediated disruption of SWRs and may account for cannabinoid‐induced impairment of hippocampus‐dependent memory.

Anatomical funneling, sparse connectivity and redundancy reduction in the neural networks of the basal ganglia

The major anatomical characteristics of the main axis of the basal ganglia are: (1) Numerical reduction in the number of neurons across layers of the feed-forward network, (2) lateral inhibitory connections within the layers, and (3) neuro-modulatory effects of dopamine and acetylcholine, both on the basal ganglia neurons and on the efficacy of information transmission along the basal ganglia axis. We recorded the simultaneous activity of neurons in the output stages of the basal ganglia as well as the activity of dopaminergic and cholinergic neurons during the performance of a probability decision-making task. We found that the functional messages of the cholinergic and dopaminergic neurons differ, and that the cholinergic message is less specific than that of the dopaminergic neurons. The output stage of the basal ganglia showed uncorrelated neuronal activity. We conclude that despite the huge numerical reduction from the cortex to the output nuclei of the basal ganglia, the activity of these nuclei represents an optimally compressed (uncorrelated) version of distinctive features of cortical information.

Physiological studies of information processing in the normal and Parkinsonian basal ganglia: pallidal activity in Go/No-Go task and following MPTP treatment

Understanding the role of the basal ganglia in day to day behavior is critical for a better understanding of the role of these structures in pathological states--such as Parkinsons disease. To elucidate this connection, we studied pallidal activity in a monkey performing a delayed release Go/No-Go task and in monkeys treated with the dopaminergic neurotoxin--MPTP. We compared the results with the predictions of the action selection and reinforcement driven dimensionality reduction models of the basal ganglia. The fraction of responding pallidal neurons, as well as the ratio of positive to negative responses, were equal in the Go and the No-Go modes. The fraction of pallidal neurons with significant responses following the trigger signal (19/26) was higher than that following the visual cue (11/26); however, the fraction of negative responses was significantly higher following the cue signal (47%) than that following the trigger signal (22%). Most (80%) of the cue responses were sensitive to the laterality of the cue, whereas only 25% of the responses following the trigger signal were sensitive to the cue or movement direction. Finally, pallidal spiking activity was not correlated in the normal behaving monkey, and became highly synchronized following MPTP treatment. We conclude that pallidal activity in the normal monkey is consistent with the model of action selection, assuming that action is selected following the visual cue. However, the reinforcement driven dimensionality reduction model is consistent with both the Go/No-Go responses and the normal/MPTP correlation studies.

The effects of motivation on response rate: A hidden semi-Markov model analysis of behavioral dynamics

A central goal of neuroscience is to understand how neural dynamics bring about the dynamics of behavior. However, neural and behavioral measures are noisy, requiring averaging over trials and subjects. Unfortunately, averaging can obscure the very dynamics that we are interested in, masking abrupt changes and artificially creating gradual processes. We develop a hidden semi-Markov model for precisely characterizing dynamic processes and their alteration due to experimental manipulations. This method takes advantage of multiple trials and subjects without compromising the information available in individual events within a trial. We apply our model to studying the effects of motivation on response rates, analyzing data from hungry and sated rats trained to press a lever to obtain food rewards on a free-operant schedule. Our method can accurately account for punctate changes in the rate of responding and for sequential dependencies between responses. It is ideal for inferring the statistics of underlying response rates and the probability of switching from one response rate to another. Using the model, we show that hungry rats have more distinct behavioral states that are characterized by high rates of responding and they spend more time in these high-press-rate states. Moreover, hungry rats spend less time in, and have fewer distinct states that are characterized by a lack of responding (Waiting/Eating states). These results demonstrate the utility of our analysis method, and provide a precise quantification of the effects of motivation on response rates.