Dezhe Z. Jin

Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories

Learning to perform a behavioural procedure as a well-ingrained habit requires extensive repetition of the behavioural sequence, and learning not to perform such behaviours is notoriously difficult. Yet regaining a habit can occur quickly, with even one or a few exposures to cues previously triggering the behaviour. To identify neural mechanisms that might underlie such learning dynamics, we made long-term recordings from multiple neurons in the sensorimotor striatum, a basal ganglia structure implicated in habit formation, in rats successively trained on a reward-based procedural task, given extinction training and then given reacquisition training. The spike activity of striatal output neurons, nodal points in cortico-basal ganglia circuits, changed markedly across multiple dimensions during each of these phases of learning. First, new patterns of task-related ensemble firing successively formed, reversed and then re-emerged. Second, task-irrelevant firing was suppressed, then rebounded, and then was suppressed again. These changing spike activity patterns were highly correlated with changes in behavioural performance. We propose that these changes in task representation in cortico-basal ganglia circuits represent neural equivalents of the explore–exploit behaviour characteristic of habit learning.

Support for a synaptic chain model of neuronal sequence generation

In songbirds, the remarkable temporal precision of song is generated by a sparse sequence of bursts in the premotor nucleus HVC. To distinguish between two possible classes of models of neural sequence generation, we carried out intracellular recordings of HVC neurons in singing zebra finches (Taeniopygia guttata). We found that the subthreshold membrane potential is characterized by a large, rapid depolarization 5–10 ms before burst onset, consistent with a synaptically connected chain of neurons in HVC. We found no evidence for the slow membrane potential modulation predicted by models in which burst timing is controlled by subthreshold dynamics. Furthermore, bursts ride on an underlying depolarization of ∼10-ms duration, probably the result of a regenerative calcium spike within HVC neurons that could facilitate the propagation of activity through a chain network with high temporal precision. Our results provide insight into the fundamental mechanisms by which neural circuits can generate complex sequential behaviours.

Neural representation of time in cortico-basal ganglia circuits

Encoding time is universally required for learning and structuring motor and cognitive actions, but how the brain keeps track of time is still not understood. We searched for time representations in cortico-basal ganglia circuits by recording from thousands of neurons in the prefrontal cortex and striatum of macaque monkeys performing a routine visuomotor task. We found that a subset of neurons exhibited time-stamp encoding strikingly similar to that required by models of reinforcement-based learning: They responded with spike activity peaks that were distributed at different time delays after single task events. Moreover, the temporal evolution of the population activity allowed robust decoding of task time by perceptron models. We suggest that time information can emerge as a byproduct of event coding in cortico-basal ganglia circuits and can serve as a critical infrastructure for behavioral learning and performance.

Intrinsic bursting enhances the robustness of a neural network model of sequence generation by avian brain area HVC

Avian brain area HVC is known to be important for the production of birdsong. In zebra finches, each RA-projecting neuron in HVC emits a single burst of spikes during a song motif. The population of neurons is activated in a precisely timed, stereotyped sequence. We propose a model of these burst sequences that relies on two hypotheses. First, we hypothesize that the sequential order of bursting is reflected in the excitatory synaptic connections between neurons. Second, we propose that the neurons are intrinsically bursting, so that burst duration is set by cellular properties. Our model generates burst sequences similar to those observed in HVC. If intrinsic bursting is removed from the model, burst sequences can also be produced. However, they require more fine-tuning of synaptic strengths, and are therefore less robust. In our model, intrinsic bursting is caused by dendritic calcium spikes, and strong spike frequency adaptation in the soma contributes to burst termination.

Acoustic sequences in non-human animals: a tutorial review and prospectus

Animal acoustic communication often takes the form of complex sequences, made up of multiple distinct acoustic units. Apart from the well‐known example of birdsong, other animals such as insects, amphibians, and mammals (including bats, rodents, primates, and cetaceans) also generate complex acoustic sequences. Occasionally, such as with birdsong, the adaptive role of these sequences seems clear (e.g. mate attraction and territorial defence). More often however, researchers have only begun to characterise – let alone understand – the significance and meaning of acoustic sequences. Hypotheses abound, but there is little agreement as to how sequences should be defined and analysed. Our review aims to outline suitable methods for testing these hypotheses, and to describe the major limitations to our current and near‐future knowledge on questions of acoustic sequences. This review and prospectus is the result of a collaborative effort between 43 scientists from the fields of animal behaviour, ecology and evolution, signal processing, machine learning, quantitative linguistics, and information theory, who gathered for a 2013 workshop entitled, ‘Analysing vocal sequences in animals’. Our goal is to present not just a review of the state of the art, but to propose a methodological framework that summarises what we suggest are the best practices for research in this field, across taxa and across disciplines. We also provide a tutorial‐style introduction to some of the most promising algorithmic approaches for analysing sequences. We divide our review into three sections: identifying the distinct units of an acoustic sequence, describing the different ways that information can be contained within a sequence, and analysing the structure of that sequence. Each of these sections is further subdivided to address the key questions and approaches in that area. We propose a uniform, systematic, and comprehensive approach to studying sequences, with the goal of clarifying research terms used in different fields, and facilitating collaboration and comparative studies. Allowing greater interdisciplinary collaboration will facilitate the investigation of many important questions in the evolution of communication and sociality.

Development of Neural Circuitry for Precise Temporal Sequences through Spontaneous Activity, Axon Remodeling, and Synaptic Plasticity

Temporally precise sequences of neuronal spikes that span hundreds of milliseconds are observed in many brain areas, including songbird premotor nucleus, cat visual cortex, and primary motor cortex. Synfire chains—networks in which groups of neurons are connected via excitatory synapses into a unidirectional chain—are thought to underlie the generation of such sequences. It is unknown, however, how synfire chains can form in local neural circuits, especially for long chains. Here, we show through computer simulation that long synfire chains can develop through spike-time dependent synaptic plasticity and axon remodeling—the pruning of prolific weak connections that follows the emergence of a finite number of strong connections. The formation process begins with a random network. A subset of neurons, called training neurons, intermittently receive superthreshold external input. Gradually, a synfire chain emerges through a recruiting process, in which neurons within the network connect to the tail of the chain started by the training neurons. The model is robust to varying parameters, as well as natural events like neuronal turnover and massive lesions. Our model suggests that long synfire chain can form during the development through self-organization, and axon remodeling, ubiquitous in developing neural circuits, is essential in the process.

Alteration of Visual Input Results in a Coordinated Reorganization of Multiple Visual Cortex Maps

In the adult visual cortex, multiple feature maps exist and have characteristic spatial relationships with one another. The relationships can be reproduced by “dimension-reduction” computational models, suggesting that the principles of continuity and coverage may underlie cortical map organization. However, the mechanisms responsible for establishing these relationships are unknown. We explored whether removing one feature map during development causes a coordinated reorganization of the remaining maps or whether the remaining maps are unaffected. We removed the ocular dominance map by monocular enucleation in newborn ferrets, so that single eye stimulation drove the cortex in a more spatially uniform manner in adult monocular animals compared with normal animals. Maps of orientation, spatial frequency, and retinotopy formed in monocular ferrets, but their structures and spatial relationships differed from those in normal ferrets. The wavelength of the orientation map increased, so that the average orientation gradient across the cortex decreased. The decrease in the orientation gradient in monocular animals was most prominent in the high gradient regions of the spatial frequency map, indicating a coordinated reorganization between these two maps. In monocular animals, the orthogonal relationship between the orientation and spatial frequency maps was preserved, and the orthogonal relationship between the orientation and retinotopic maps became more pronounced. These results were consistent with detailed predictions of a dimension-reduction model of cortical organization. Thus, the number of feature maps in a cortical area influences the relationships between them, and inputs to the cortex have a significant role in generating these relationships.

A Compact Statistical Model of the Song Syntax in Bengalese Finch

Songs of many songbird species consist of variable sequences of a finite number of syllables. A common approach for characterizing the syntax of these complex syllable sequences is to use transition probabilities between the syllables. This is equivalent to the Markov model, in which each syllable is associated with one state, and the transition probabilities between the states do not depend on the state transition history. Here we analyze the song syntax in Bengalese finch. We show that the Markov model fails to capture the statistical properties of the syllable sequences. Instead, a state transition model that accurately describes the statistics of the syllable sequences includes adaptation of the self-transition probabilities when states are revisited consecutively, and allows associations of more than one state to a given syllable. Such a model does not increase the model complexity significantly. Mathematically, the model is a partially observable Markov model with adaptation (POMMA). The success of the POMMA supports the branching chain network model of how syntax is controlled within the premotor song nucleus HVC, but also suggests that adaptation and many-to-one mapping from the syllable-encoding chain networks in HVC to syllables should be included in the network model.

Animal vocal sequences: not the Markov chains we thought they were.

Many animals produce vocal sequences that appear complex. Most researchers assume that these sequences are well characterized as Markov chains (i.e. that the probability of a particular vocal element can be calculated from the history of only a finite number of preceding elements). However, this assumption has never been explicitly tested. Furthermore, it is unclear how language could evolve in a single step from a Markovian origin, as is frequently assumed, as no intermediate forms have been found between animal communication and human language. Here, we assess whether animal taxa produce vocal sequences that are better described by Markov chains, or by non-Markovian dynamics such as the ‘renewal process’ (RP), characterized by a strong tendency to repeat elements. We examined vocal sequences of seven taxa: Bengalese finches Lonchura striata domestica, Carolina chickadees Poecile carolinensis, free-tailed bats Tadarida brasiliensis, rock hyraxes Procavia capensis, pilot whales Globicephala macrorhynchus, killer whales Orcinus orca and orangutans Pongo spp. The vocal systems of most of these species are more consistent with a non-Markovian RP than with the Markovian models traditionally assumed. Our data suggest that non-Markovian vocal sequences may be more common than Markov sequences, which must be taken into account when evaluating alternative hypotheses for the evolution of signalling complexity, and perhaps human language origins.

Optimal habits can develop spontaneously through sensitivity to local cost

Habits and rituals are expressed universally across animal species. These behaviors are advantageous in allowing sequential behaviors to be performed without cognitive overload, and appear to rely on neural circuits that are relatively benign but vulnerable to takeover by extreme contexts, neuropsychiatric sequelae, and processes leading to addiction. Reinforcement learning (RL) is thought to underlie the formation of optimal habits. However, this theoretic formulation has principally been tested experimentally in simple stimulus-response tasks with relatively few available responses. We asked whether RL could also account for the emergence of habitual action sequences in realistically complex situations in which no repetitive stimulus-response links were present and in which many response options were present. We exposed naïve macaque monkeys to such experimental conditions by introducing a unique free saccade scan task. Despite the highly uncertain conditions and no instruction, the monkeys developed a succession of stereotypical, self-chosen saccade sequence patterns. Remarkably, these continued to morph for months, long after session-averaged reward and cost (eye movement distance) reached asymptote. Prima facie, these continued behavioral changes appeared to challenge RL. However, trial-by-trial analysis showed that pattern changes on adjacent trials were predicted by lowered cost, and RL simulations that reduced the cost reproduced the monkeys’ behavior. Ultimately, the patterns settled into stereotypical saccade sequences that minimized the cost of obtaining the reward on average. These findings suggest that brain mechanisms underlying the emergence of habits, and perhaps unwanted repetitive behaviors in clinical disorders, could follow RL algorithms capturing extremely local explore/exploit tradeoffs.