Antonius B. Mulder

Learning-related changes in response patterns of prefrontal neurons during instrumental conditioning.

A crucial aspect of organizing goal-directed behavior is the ability to form neural representations of relationships between environmental stimuli, actions and reinforcement. Very little is known yet about the neural encoding of response-reward relationships, a process which is deemed essential for purposeful behavior. To investigate this, tetrode recordings were made in the medial prefrontal cortex (PFC) of rats performing a Go-NoGo task. After task acquisition, a subset of neurons showed a sustained change in firing during the rewarded action sequence that was triggered by a specific visual cue. When these changes were monitored in the course of learning, they were seen to develop in parallel with the behavioral learning curve and were highly sensitive to a switch in reward contingencies. These sustained changes correlated with the reward-associated action sequence, not with sensory or reward-predicting properties of the cue or individual motor acts per se. This novel type of neural plasticity may contribute to the formation of response-reinforcer associations and of behavioral strategies for guiding goal-directed action.

Position and behavioral modulation of synchronization of hippocampal and accumbens neuronal discharges in freely moving rats.

To understand how hippocampal signals are processed by downstream neurons, we analyzed the relative timing between neuronal discharges in simultaneous recordings in the hippocampus and nucleus accumbens of rats performing in a plus maze. In all, 154 pairs of cells (composed of 65 hippocampal and 56 accumbens neurons) were examined during the 1 s period prior to reward delivery. Cross‐correlation analyses over a ±300‐ms window with 10‐ms bins revealed that 108 pairs had at least one significant histogram bin (P < 0.01). The most frequently occurring peaks of hippocampal firing prior to accumbens discharges appeared at latencies from −30–0 ms, corresponding to published values of the latency of the hippocampal pathway to the nucleus accumbens. Other peaks appeared most often at latencies multiples of about 110 ms prior to and after this, corresponding to theta rhythmicity. Since firing synchronization can result from several types of connectivity patterns (such as common inputs), a group of 18 hippocampus‐accumbens pairs was selected as those most likely to have monosynaptic connections. The criterion was the presence of at least one highly significant peak (P < 0.001) at latencies corresponding to field potentials evoked in the accumbens by hippocampal stimulation. A significant peak occurred on all four maze arms for only one of these cell pairs, indicating positional modulation for the others. In addition, behavior dependence of the synchrony between these nucleus accumbens and hippocampus neurons was examined by studying data in relation to three different synchronization points: reward box arrival, box departure, and arrival at the center of the maze. This indicates that the functional connectivity between hippocampal and accumbens neurons was stronger when the rat was near reward areas. Ten of the hippocampal neurons in these 18 cell pairs showed 9‐Hz (theta) rhythmic activity in autocorrelation analyses. Of these 10 cells, cross‐correlograms from eight hippocampal‐accumbens pairs also showed theta rhythmicity. Overall, these results indicate that the synchrony between hippocampus and nucleus accumbens neurons is modulated by spatial position and behavior, and theta rhythm may play an important role for this synchronization. Hippocampus 2000;10:717–728.

Pharmacological Manipulation of Neuronal Ensemble Activity by Reverse Microdialysis in Freely Moving Rats: A Comparative Study of the Effects of Tetrodotoxin, Lidocaine, and Muscimol

To be able to address the question how neurotransmitters or pharmacological agents influence activity of neuronal populations in freely moving animals, the combidrive was developed. The combidrive combines an array of 12 tetrodes to perform ensemble recordings with a moveable and replaceable microdialysis probe to locally administer pharmacological agents. In this study, the effects of cumulative concentrations of tetrodotoxin, lidocaine, and muscimol on neuronal firing activity in the prefrontal cortex were examined and compared. These drugs are widely used in behavioral studies to transiently inactivate brain areas, but little is known about their effects on ensemble activity and the possible differences between them. The results show that the combidrive allows ensemble recordings simultaneously with reverse microdialysis in freely moving rats for periods at least up to 2 wk. All drugs reduced neuronal firing in a concentration dependent manner, but they differed in the extent to which firing activity of the population was decreased and the in speed and extent of recovery. At the highest concentration used, both muscimol and tetrodotoxin (TTX) caused an almost complete reduction of firing activity. Lidocaine showed the fastest recovery, but it resulted in a smaller reduction of firing activity of the population. From these results, it can be concluded that whenever during a behavioral experiment a longer lasting, reversible inactivation is required, muscimol is the drug of choice, because it inactivates neurons to a similar degree as TTX, but it does not, in contrast to TTX, affect fibers of passage. For a short-lasting but partial inactivation, lidocaine would be most suitable.

Lesions of the medial shell of the nucleus accumbens impair rats in finding larger rewards, but spare reward-seeking behavior.

The goal of this study was to help better understand the importance of the nucleus accumbens (Nacc) in the processing of position and reward value information for goal-directed orientation behaviors. Sixteen male Long-Evans rats, under partial water deprivation, were trained in a plus-maze to find water rewards in the respective arms which were lit in pseudo-random sequence (training trials). Each day one reward arm was selected to deliver six drops of water (at 1 s intervals) the others provided only one drop per visit. After 32 visits, probe trials were intermittently presented among training trials. Here, all four arms were lit and offered the previously assigned reward. The rats rapidly learned to go to the highly rewarded arm. Six trained rats were given bilateral electrolytic lesions in the Nacc shell, two others had unilateral lesions and eight had sham operations (with approved protocols). Field potentials evoked by fornix stimulation were recorded in lesion electrodes to guide placements. Only the lesioned rats showed significant impairments (P<0.05) in selecting the greater reward on probe trials. However on training trials, lesioned (and sham-operated) rats made only rare errors. While the motivation to drink and the capacity for cue-guided goal-directed orientation behavior was spared, lesioned rats were impaired in learning the location of the larger reward. The accumbens lesions apparently impaired integration of position and reward value information, consistent with anatomical and electrophysiological data showing the convergence of hippocampal, amygdalar, ventral tegmental area (VTA) and prefrontal cortical inputs there.

Position sensitivity in phasically discharging nucleus accumbens neurons of rats alternating between tasks requiring complementary types of spatial cues

To determine how hippocampal location-selective discharges might influence downstream structures for navigation, nucleus accumbens neurons were recorded in rats alternating between two tasks guided respectively by lit cues in the maze or by extramaze room cues. Of 144 phasically active neurons, 80 showed significant behavioral correlates including displacements, immobility prior to, or after reward delivery, as well as turning, similar to previous reports. Nine neurons were position-selective, 22 were sensitive to task and platform changes and 40 others were both. Although the accumbens neurons showed the same behavioral correlate in two or four functionally equivalent locations, these responses were stronger at some of these places, evidence for position sensitivity. To test whether position responses were selective for room versus platform cues, the experimental platform was rotated while the rat performed each of the two tasks. This revealed responses to changes in position relative to both platform and room cues, despite the fact that previous studies had shown that place responses of hippocampal neurons recorded in the same task are anchored to room cues only. After these manipulations and shifts between the two tasks, the responses varied among simultaneously recorded neurons, and even in single neurons in alternating visits to reward sites. Again this contrasts with the uniformity of place responses of hippocampal neurons recorded in this same task. Thus accumbens position responses may derive from hippocampal inputs, while responses to context changes are more likely to derive from other signals or intrinsic processing. Considering the accumbens as a limbic-motor interface, we conclude that position-modulated behavioral responses in the accumbens may be intermediate between the allocentric reference frame of position-selective discharges in the hippocampus and the egocentric coding required to organize movement control. The conflicting responses among simultaneously recorded neurons could reflect competition processes serving as substrates for action selection and learning.

Neurons in hippocampal afferent zones of rat striatum parse routes into multi-pace segments during maze navigation.

Hippocampal ‘place’ neurons discharge when rats occupy specific regions within an environment. This finding is a cornerstone of the theory of the hippocampus as a cognitive map of space. But for navigation, representations of current position must be implemented by signals concerning where to go next, and how to get there. In recordings in hippocampal output structures associated with the motor system (nucleus accumbens and ventromedial caudate nucleus) in rats solving a plus‐maze, neurons fired continuously from the moment the rat left one location until it arrived at the next goal site, or at an intermediate place, such as the maze centre. While other studies have shown discharges during reward approach behaviours, this is the first demonstration of activity corresponding to the parsing of complex routes into sequences of movements between landmarks, similar to the lists of instructions we often employ to communicate directions to follow between points on a map. As these cells fired during a series of several paces or re‐orientation movements, perhaps this is homologous to ‘chunking’. The temporal overlaps in the activity profiles of the individual neurons provide a possible substrate to successively trigger movements required to arrive at the goal. These hippocampally informed, and in some cases, spatially selective responses support the view of the ventral striatum as an interface between limbic and motor systems, permitting contextual representations to have an impact on fundamental action sequences for goal‐directed behaviour.

Anticipatory reward signals in ventral striatal neurons of behaving rats

It has been proposed that the striatum plays a crucial role in learning to select appropriate actions, optimizing rewards according to the principles of ‘Actor–Critic’ models of trial‐and‐error learning. The ventral striatum (VS), as Critic, would employ a temporal difference (TD) learning algorithm to predict rewards and drive dopaminergic neurons. This study examined this model’s adequacy for VS responses to multiple rewards in rats. The respective arms of a plus‐maze provided rewards of varying magnitudes; multiple rewards were provided at 1‐s intervals while the rat stood still. Neurons discharged phasically prior to each reward, during both initial approach and immobile waiting, demonstrating that this signal is predictive and not simply motor‐related. In different neurons, responses could be greater for early, middle or late droplets in the sequence. Strikingly, this activity often reappeared after the final reward, as if in anticipation of yet another. In contrast, previous TD learning models show decremental reward‐prediction profiles during reward consumption due to a temporal‐order signal introduced to reproduce accurate timing in dopaminergic reward‐prediction error signals. To resolve this inconsistency in a biologically plausible manner, we adapted the TD learning model such that input information is nonhomogeneously distributed among different neurons. By suppressing reward temporal‐order signals and varying richness of spatial and visual input information, the model reproduced the experimental data. This validates the feasibility of a TD‐learning architecture where different groups of neurons participate in solving the task based on varied input information.

Plasticity of neuronal firing in deep layers of the medial prefrontal cortex in rats engaged in operant conditioning.

Publisher Summary Although many studies, especially in primates, have focused on the issue of associative learning processes and studied working memory, detailed information on how neuronal representations of learned associations are actually formed is still largely lacking. Therefore, the emphasis of this chapter lies on neural plasticity during the formation of learned associations and executive functions of the prefrontal cortex (PFC) focusing on the relationship between changing behaviors correlating with plasticity of neuronal firing in rats performing a discrimination, Go-NoGo, task. Before going into details of the neuronal representation of learned associations in awake animals, the chapter discusses the variety of identified behavioral correlates of PFC neurons in the context of established performance during PFC-dependent operant conditioning tasks. Thereafter, synaptic plasticity in the PFC as a model for learning and memory processes is discussed; moreover, the functional relevance of the findings is also discussed.

Hippocampal neuronal position selectivity remains fixed to room cues only in rats alternating between place navigation and beacon approach tasks

To study the relationship between brain representations and behaviour, we recorded hippocampal neuronal activity in rats repeatedly alternating between two different tasks on a circular platform with four reward boxes along the edge. In the beacon approach task, rewards were provided only at the pair of diametrically opposite boxes that was illuminated. In the place navigation task, rewards were available only at the boxes positioned near the north‐east and south‐west corners of the room. Performance levels were high and rats rapidly reoriented to changes in lamp cues in the beacon approach task. Neuropsychological studies show that rats with hippocampal lesions readily employ beacon approach strategies, while place navigation is severely impaired. Previous studies suggested that the neurons might change their behavioural correlates as the rat performed the respective tasks. However, of 34 hippocampal ‘place cells’ recorded, all showed position selectivity fixed with respect to room cues, even in the beacon approach task where coding the position of the rat in the room was of no use for locating rewards. Whether or not hippocampal signals are actually employed for ongoing behaviour would then be decided by structures downstream from the hippocampus. If this is the case, then the ‘counterproductive’ room referred place‐related discharges in the beacon approach task would be a background representation. This would provide support for proposals of multiple memory systems underlying different types of information processing and contrasts with the popular notion that local neuronal activity levels are selectively increased to the degree that the brain region is required for the ongoing function.

Spatial and behavioral correlates in nucleus accumbens neurons in zones receiving hippocampal or prefrontal cortical inputs

Abstract In order to better understand the impact of hippocampal processing on downstream neural structures and cognitive functions, neurons were simultaneously recorded in the hippocampus and in basal ganglia zones receiving inputs from hippocampus directly (the nucleus accumbens shell) or indirectly via the prefrontal cortex (nucleus accumbens core and ventromedial caudate nucleus) in rats performing spatial orientation tasks. In one series of experiments, the animals alternated between using intramaze and extramaze cues to find water rewards. In a second series, the rats were required to learn the distribution of different reward quantities provided at the respective goal boxes in a plus maze. Correct performance in the latter task has been shown to be impaired by lesions of the accumbens shell. Hippocampal place responses were anchored to the extramaze cues and were independent of reward values provided near or in the firing fields. While no hippocampal-like firing fields were found in accumbens neurons, neuronal activity during pre-reward, post-reward and reward approach behaviors was more intense at some locations than at others. This is consistent with anatomical and physiological observations corresponding to the convergence of hippocampal position information with reward related signals from the amygdala and ventral tegmental area (VTA).