Matthew J. Crossley

A computational model of how cholinergic interneurons protect striatal-dependent learning

An essential component of skill acquisition is learning the environmental conditions in which that skill is relevant. This article proposes and tests a neurobiologically detailed theory of how such learning is mediated. The theory assumes that a key component of this learning is provided by the cholinergic interneurons in the striatum known as tonically active neurons (TANs). The TANs are assumed to exert a tonic inhibitory influence over cortical inputs to the striatum that prevents the execution of any striatal-dependent actions. The TANs learn to pause in rewarding environments, and this pause releases the striatal output neurons from this inhibitory effect, thereby facilitating the learning and expression of striatal-dependent behaviors. When rewards are no longer available, the TANs cease to pause, which protects striatal learning from decay. A computational version of this theory accounts for a variety of single-cell recording data and some classic behavioral phenomena, including fast reacquisition after extinction.

Interactions between declarative and procedural-learning categorization systems.

Two experiments tested whether declarative and procedural memory systems operate independently or inhibit each other during perceptual categorization. Both experiments used a hybrid category-learning task in which perfect accuracy could be achieved if a declarative strategy is used on some trials and a procedural strategy is used on others. In the two experiments, only 2 of 53 participants learned a strategy of this type. In Experiment 1, most participants appeared to use simple explicit rules, even though control participants reliably learned the procedural component of the hybrid task. In Experiment 2, participants pre-trained either with the declarative or procedural component and then transferred to the hybrid categories. Despite this extra training, no participants in either group learned to categorize the hybrid stimuli with a strategy of the optimal type. These results are inconsistent with the most prominent single- and multiple-system accounts of category learning. They also cannot be explained by knowledge partitioning, or by the hypothesis that the failure to learn was due to high switch costs. Instead, these results support the hypothesis that declarative and procedural memory systems interact during category learning.

Savings upon re-aiming in visuomotor adaptation

Sensorimotor adaptation has traditionally been viewed as a purely error-based process. There is, however, growing appreciation for the idea that performance changes in these tasks can arise from the interplay of error-based adaptation with other learning processes. The challenge is to specify constraints on these different processes, elucidating their respective contributions to performance, as well as the manner in which they interact. We address this question by exploring constraints on savings, the phenomenon in which people show faster performance gains when the same learning task is repeated. In a series of five experiments, we demonstrate that error-based learning associated with sensorimotor adaptation does not contribute to savings. Instead, savings reflects improvements in action selection, rather than motor execution. SIGNIFICANCE STATEMENT Savings is the phenomenon in which people show faster relearning of a previously forgotten memory. In the motor learning domain, this phenomenon has been a puzzle for learning models that operate exclusively on error-based learning processes. We demonstrate, in a series of experiments, that savings selectively reflects improvements in action selection: Participants are more adept in invoking an appropriate aiming strategy when presented with a previously experienced perturbation. Indeed, improvements in action selection appear to be the sole source of savings in visuomotor adaptation tasks. We observe no evidence of savings in implicit error-based adaptation.

Automaticity and multiple memory systems.

A large number of criteria have been proposed for determining when a behavior has become automatic. Almost all of these were developed before the widespread acceptance of multiple memory systems. Consequently, popular frameworks for studying automaticity often neglect qualitative differences in how different memory systems guide initial learning. Unfortunately, evidence suggests that automaticity criteria derived from these frameworks consistently misclassify certain sets of initial behaviors as automatic. Specifically, criteria derived from cognitive science mislabel much behavior still under the control of procedural memory as automatic, and criteria derived from animal learning mislabel some behaviors under the control of declarative memory as automatic. Even so, neither set of criteria make the opposite error-that is, both sets correctly identify any automatic behavior as automatic. In fact, evidence suggests that although there are multiple memory systems and therefore multiple routes to automaticity, there might nevertheless be only one common representation for automatic behaviors. A number of possible cognitive and cognitive neuroscience models of this single automaticity system are reviewed. WIREs Cogn Sci 2012, 3:363-376. doi: 10.1002/wcs.1172 For further resources related to this article, please visit the WIREs website.

Erasing the engram: the unlearning of procedural skills.

Huge amounts of money are spent every year on unlearning programs--in drug-treatment facilities, prisons, psychotherapy clinics, and schools. Yet almost all of these programs fail, since recidivism rates are high in each of these fields. Progress on this problem requires a better understanding of the mechanisms that make unlearning so difficult. Much cognitive neuroscience evidence suggests that an important component of these mechanisms also dictates success on categorization tasks that recruit procedural learning and depend on synaptic plasticity within the striatum. A biologically detailed computational model of this striatal-dependent learning is described (based on Ashby & Crossley, 2011). The model assumes that a key component of striatal-dependent learning is provided by interneurons in the striatum called the tonically active neurons (TANs), which act as a gate for the learning and expression of striatal-dependent behaviors. In their tonically active state, the TANs prevent the expression of any striatal-dependent behavior. However, they learn to pause in rewarding environments and thereby permit the learning and expression of striatal-dependent behaviors. The model predicts that when rewards are no longer contingent on behavior, the TANs cease to pause, which protects striatal learning from decay and prevents unlearning. In addition, the model predicts that when rewards are partially contingent on behavior, the TANs remain partially paused, leaving the striatum available for unlearning. The results from 3 human behavioral studies support the model predictions and suggest a novel unlearning protocol that shows promising initial signs of success.

Multiple stages of learning in perceptual categorization: evidence and neurocomputational theory

Virtually all current theories of category learning assume that humans learn new categories by gradually forming associations directly between stimuli and responses. In information-integration category-learning tasks, this purported process is thought to depend on procedural learning implemented via dopamine-dependent cortical-striatal synaptic plasticity. This article proposes a new, neurobiologically detailed model of procedural category learning that, unlike previous models, does not assume associations are made directly from stimulus to response. Rather, the traditional stimulus-response (S-R) models are replaced with a two-stage learning process. Multiple streams of evidence (behavioral, as well as anatomical and fMRI) are used as inspiration for the new model, which synthesizes evidence of multiple distinct cortical-striatal loops into a neurocomputational theory. An experiment is reported to test a priori predictions of the new model that: (1) recovery from a full reversal should be easier than learning new categories equated for difficulty, and (2) reversal learning in procedural tasks is mediated within the striatum via dopamine-dependent synaptic plasticity. The results confirm the predictions of the new two-stage model and are incompatible with existing S-R models.

Procedural learning of unstructured categories

Unstructured categories are those in which the stimuli are assigned to each contrasting category randomly, and thus there is no rule- or similarity-based strategy for determining category membership. Intuition suggests that unstructured categories are likely to be learned via explicit memorization that is under the control of declarative memory. In contrast to this prediction, neuroimaging studies of unstructured-category learning have reported task-related activation in the striatum, but typically not in the hippocampus—results that seem more consistent with procedural learning than with a declarative-memory strategy. This article reports the first known behavioral test of whether unstructured-category learning is mediated by explicit strategies or by procedural learning. Our results suggest that the feedback-based learning of unstructured categories is mediated by procedural memory.

Context-dependent savings in procedural category learning.

Environmental context can have a profound influence on the efficacy of intervention protocols designed to eliminate undesirable behaviors. This is clearly seen in drug rehabilitation clinics where patients often relapse soon after leaving the context of the treatment facility. A similar pattern is commonly observed in controlled laboratory studies of context-dependent savings in instrumental conditioning, where simply placing an animal back into the original conditioning chamber can renew an extinguished instrumental response. Surprisingly, context-dependent savings in human procedural learning has not been carefully examined in the laboratory. Here, we provide the first known empirical demonstration of context-dependent savings in a perceptual categorization task known to recruit procedural learning. We also present a computational account of these savings using a biologically detailed model in which a key role is played by cholinergic interneurons in the striatum.

Procedural Learning during Declarative Control.

There is now abundant evidence that human learning and memory are governed by multiple systems. As a result, research is now turning to the next question of how these putative systems interact. For instance, how is overall control of behavior coordinated, and does learning occur independently within systems regardless of what system is in control? Behavioral, neuroimaging, and neuroscience data are somewhat mixed with respect to these questions. Human neuroimaging and animal lesion studies suggest independent learning and are mostly agnostic with respect to control. Human behavioral studies suggest active inhibition of behavioral output but have little to say regarding learning. The results of two perceptual category-learning experiments are described that strongly suggest that procedural learning does occur while the explicit system is in control of behavior and that this learning might be just as good as if the procedural system was controlling the response. These results are consistent with the idea that declarative memory systems inhibit the ability of the procedural system to access motor output systems but do not prevent procedural learning.

Expanding the role of striatal cholinergic interneurons and the midbrain dopamine system in appetitive instrumental conditioning

The basal ganglia are a collection of subcortical nuclei thought to underlie a wide variety of vertebrate behavior. Although a great deal is known about the functional and physiological properties of the basal ganglia, relatively few models have been formally developed that have been tested against both behavioral and physiological data. Our previous work (Ashby FG, Crossley MJ. J Cogn Neurosci 23: 1549-1566, 2011) showed that a model grounded in the neurobiology of the basal ganglia could account for basic single-neuron recording data, as well as behavioral phenomena such as fast reacquisition that constrain models of conditioning. In this article we show that this same model accounts for a variety of appetitive instrumental conditioning phenomena, including the partial reinforcement extinction (PRE) effect, rapid and slowed reacquisition following extinction, and renewal of previously extinguished instrumental responses by environmental context cues.