Henry W. Chase

Failure to Consolidate the Consolidation Theory of Learning for Sensorimotor Adaptation Tasks

An influential idea in human motor learning is that there is a consolidation period during which motor memories are transformed from a fragile to a permanent state, no longer susceptible to interference from new learning. The evidence supporting this idea comes from studies showing that the motor memory of a task (A) is lost when an opposing task (B) is experienced soon after, but not if sufficient time is allowed to pass (∼6 hr). We report results from three laboratories challenging this consolidation idea. We used an ABA paradigm in the context of a reaching task to assess the influence of experiencing B after A on the retention of A. In two experiments using visuomotor rotations, we found that B fully interferes with the retention of A even when B is experienced 24 hr after A. Contrary to previous reports, in four experiments on learning force fields, we also observed full interference between A and B when they are separated by 24 hr or even 1 week. This latter result holds for both position-dependent and velocity-dependent force fields. For both the visuomotor and force-field tasks, complete interference is still observed when the possible affects of anterograde interference are controlled through the use of washout trials. Our results fail to support the idea that motor memories become consolidated into a protected state. Rather, they are consistent with recent ideas of memory formation, which propose that memories can shift between active and inactive states.

The neural basis of drug stimulus processing and craving: an activation likelihood estimation meta-analysis.

BACKGROUND The capacity of drug cues to elicit drug-seeking behavior is believed to play a fundamental role in drug dependence; yet the neurofunctional basis of human drug cue-reactivity is not fully understood. We performed a meta-analysis to identify brain regions that are consistently activated by presentation of drug cues. Studies involving treatment-seeking and nontreatment-seeking substance users were contrasted to determine whether there were consistent differences in the neural response to drug cues between these populations. Finally, to assess the neural basis of craving, consistency across studies in brain regions that show correlated activation with craving was assessed. METHODS Appropriate studies, assessing the effect of drug-related cues or manipulations of drug craving in drug-user populations across the whole brain, were obtained via the PubMed database and literature search. Activation likelihood estimation, a method of quantitative meta-analysis that estimates convergence across experiments by modeling the spatial uncertainty of neuroimaging data, was used to identify consistent regions of activation. RESULTS Cue-related activation was observed in the ventral striatum (across both subgroups), amygdala (in the treatment-seeking subgroup and overall), and orbitofrontal cortex (in the nontreatment-seeking subgroup and overall) but not insula cortex. Although a different pattern of frontal and temporal lobe activation between the subgroups was observed, these differences were not significant. Finally, right amygdala and left middle frontal gyrus activity were positively associated with craving. CONCLUSIONS These results substantiate the key neural substrates underlying reactivity to drug cues and drug craving.

TRPM8 activation by menthol, icilin, and cold is differentially modulated by intracellular pH

TRPM8 is a nonselective cation channel activated by cold and the cooling compounds menthol and icilin (Peier et al., 2002). Here, we have used electrophysiology and the calcium-sensitive dye Fura-2 to study the effect of pH and interactions between temperature, pH, and the two chemical agonists menthol and icilin on TRPM8 expressed in Chinese hamster ovary cells. Menthol, icilin, and cold all evoked stimulus-dependent [Ca2+]i responses in standard physiological solutions of pH 7.3. Increasing the extracellular [H+] from pH 7.3 to approximately pH 6 abolished responses to icilin and cold stimulation but did not affect responses to menthol. Icilin concentration-response curves were significantly shifted to the right when pH was lowered from 7.3 to 6.9, whereas those with menthol were unaltered in solutions of pH 6.1. When cells were exposed to solutions in the range of pH 8.1-6.5, the temperature threshold for activation was elevated at higher pH and depressed at lower pH. Superfusing cells with a low subactivating concentration of icilin or menthol elevated the threshold for cold activation at pH 7.4, but cooling failed to evoke [Ca2+]i responses at pH 6 in the presence of either agonist. In voltage-clamp experiments in which the intracellular pH was buffered to different levels, acidification reduced the current amplitude of icilin responses and shifted the threshold for cold activation to lower values with half-maximal inhibition at pH 7.2 and pH 7.6. The results demonstrate that the activation of TRPM8 by icilin and cold, but not menthol, is modulated by intracellular pH in the physiological range. Furthermore, our data suggest that activation by icilin and cold involve a different mechanism to activation by menthol.

Gambling Severity Predicts Midbrain Response to Near-Miss Outcomes

Gambling is a common recreational activity that becomes dysfunctional in a subset of individuals, with DSM “pathological gambling” regarded as the most severe form. During gambling, players experience a range of cognitive distortions that promote an overestimation of the chances of winning. Near-miss outcomes are thought to fuel these distortions. We observed previously that near misses recruited overlapping circuitry to monetary wins in a study in healthy volunteers (Clark et al., 2009). The present study sought to extend these observations in regular gamblers and relate brain responses to an index of gambling severity. Twenty regular gamblers, who varied in their involvement from recreational playing to probable pathological gambling, were scanned while performing a simplified slot machine task that delivered occasional monetary wins, as well as near-miss and full-miss nonwin outcomes. In the overall group, near-miss outcomes were associated with a significant response in the ventral striatum, which was also recruited by monetary wins. Gambling severity, measured with the South Oaks Gambling Screen, predicted a greater response in the dopaminergic midbrain to near-miss outcomes. This effect survived controlling for clinical comorbidities that were present in the regular gamblers. Gambling severity did not predict win-related responses in the midbrain or elsewhere. These results demonstrate that near-miss events during gambling recruit reward-related brain circuitry in regular players. An association with gambling severity in the midbrain suggests that near-miss outcomes may enhance dopamine transmission in disordered gambling, which extends neurobiological similarities between pathological gambling and drug addiction.

Feedback-related negativity codes prediction error but not behavioral adjustment during probabilistic reversal learning

We assessed electrophysiological activity over the medial frontal cortex (MFC) during outcome-based behavioral adjustment using a probabilistic reversal learning task. During recording, participants were presented two abstract visual patterns on each trial and had to select the stimulus rewarded on 80% of trials and to avoid the stimulus rewarded on 20% of trials. These contingencies were reversed frequently during the experiment. Previous EEG work has revealed feedback-locked electrophysiological responses over the MFC (feedback-related negativity; FRN), which correlate with the negative prediction error [Holroyd, C. B., & Coles, M. G. The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679–709, 2002] and which predict outcome-based adjustment of decision values [Cohen, M. X., & Ranganath, C. Reinforcement learning signals predict future decisions. Journal of Neuroscience, 27, 371–378, 2007]. Unlike previous paradigms, our paradigm enabled us to disentangle, on the one hand, mechanisms related to the reward prediction error, derived from reinforcement learning (RL) modeling, and on the other hand, mechanisms related to explicit rule-based adjustment of actual behavior. Our results demonstrate greater FRN amplitudes with greater RL model-derived prediction errors. Conversely expected negative outcomes that preceded rule-based behavioral reversal were not accompanied by an FRN. This pattern contrasted remarkably with that of the P3 amplitude, which was significantly greater for expected negative outcomes that preceded rule-based behavioral reversal than for unexpected negative outcomes that did not precede behavioral reversal. These data suggest that the FRN reflects prediction error and associated RL-based adjustment of decision values, whereas the P3 reflects adjustment of behavior on the basis of explicit rules.

Parallel goal-directed and habitual control of human drug-seeking: implications for dependence vulnerability.

Dual-process theories of learning and addiction propose that whereas freely elected drug/reward-seeking is goal-directed in being mediated by the expected value of the outcome, cue-elicited drug/reward-seeking is habitual in being elicited directly by antecedent stimuli, without retrieving a representation of outcome value. To substantiate this claim, the current study conducted a human devaluation-transfer procedure in which young adult smokers were first trained on a concurrent choice task to earn tobacco and chocolate points before one outcome was devalued by specific satiety or health warnings against consumption of that outcome. When choice was again tested in extinction, the selective reduction in performance of the action associated with the devalued outcome indicated that choice was controlled by an expectation of outcome value, that is, was goal-directed. Moreover, the presentation of tobacco and chocolate cues enhanced selection of the response associated with that outcome, indicating that transfer was also mediated by the retrieval of the outcome representation. Paradoxically, however, the magnitude of this transfer effect was unaffected by devaluation, indicating that the stimulus retrieved a representation of outcome identity but not current incentive value. Individual differences in tobacco dependence in the young adult sample were associated with tobacco preference in the concurrent choice task but not with the devaluation or transfer effects. These data accord with dual-process theories in suggesting that drug/reward-seeking are mediated by goal-directed and habitual controllers under freely elected and cued conditions, respectively, and that initial uptake of drug use is associated with hyper-valuation of the drug as an outcome of goal-directed drug-seeking rather than with accelerated habit formation.

Approach and avoidance learning in patients with major depression and healthy controls: relation to anhedonia

BACKGROUND Central to understanding of the behavioural consequences of depression has been the theory that the disorder is accompanied by an increased sensitivity to negative compared with positive reinforcement (negative bias), whereas other theorists have emphasized a global reduction in sensitivity to reinforcement in depression (blunting). METHOD In this study, we used a probabilistic selection task that was designed to examine independently rates of learning to predict both positive and negative reinforcement. Twenty-three depressed out-patients and 23 healthy controls from the local population participated in the study. RESULTS No evidence for a negative bias was observed on the task, either during acquisition of the task or during generalization of the learned information. Depressed patients responded slower on the task than controls but showed a similar modulation of reaction times (RTs) as controls following reinforcement. Evidence for blunting was observed on the training phase, as reflected in reduced trial-by-trial adjustment during this phase. However, this effect was related specifically to the severity of anhedonia, as measured by the Snaith-Hamilton Pleasure Scale (SHAPS), and was independent of overall depression severity. CONCLUSIONS We argue that the observation of a negative bias or blunting in a group of depressed patients may be dependent on the neuropsychological task and the symptoms of the patients tested. Our results provide insight into how these theories might be further tested.

Dissociable patterns of abnormal frontal cortical activation during anticipation of an uncertain reward or loss in bipolar versus major depression

Recent research has found abnormalities in reward‐related neural activation in bipolar disorder (BD), during both manic and euthymic phases. However, reward‐related neural activation in currently depressed individuals with BD and that in currently depressed individuals with major depressive disorder (MDD) have yet to be directly compared. Here, we studied these groups, examining the neural activation elicited during a guessing task in fronto‐striatal regions identified by previous studies.

Reinforcement learning models and their neural correlates: An activation likelihood estimation meta-analysis

Reinforcement learning describes motivated behavior in terms of two abstract signals. The representation of discrepancies between expected and actual rewards/punishments—prediction error—is thought to update the expected value of actions and predictive stimuli. Electrophysiological and lesion studies have suggested that mesostriatal prediction error signals control behavior through synaptic modification of cortico-striato-thalamic networks. Signals in the ventromedial prefrontal and orbitofrontal cortex are implicated in representing expected value. To obtain unbiased maps of these representations in the human brain, we performed a meta-analysis of functional magnetic resonance imaging studies that had employed algorithmic reinforcement learning models across a variety of experimental paradigms. We found that the ventral striatum (medial and lateral) and midbrain/thalamus represented reward prediction errors, consistent with animal studies. Prediction error signals were also seen in the frontal operculum/insula, particularly for social rewards. In Pavlovian studies, striatal prediction error signals extended into the amygdala, whereas instrumental tasks engaged the caudate. Prediction error maps were sensitive to the model-fitting procedure (fixed or individually estimated) and to the extent of spatial smoothing. A correlate of expected value was found in a posterior region of the ventromedial prefrontal cortex, caudal and medial to the orbitofrontal regions identified in animal studies. These findings highlight a reproducible motif of reinforcement learning in the cortico-striatal loops and identify methodological dimensions that may influence the reproducibility of activation patterns across studies.

Impaired goal-directed behavioural control in human impulsivity

Two dissociable learning processes underlie instrumental behaviour. Whereas goal-directed behaviour is controlled by knowledge of the consequences, habitual behaviour is elicited directly by antecedent Pavlovian stimuli without knowledge of the consequences. Predominance of habitual control is thought to underlie psychopathological conditions associated with corticostriatal abnormalities, such as impulsivity and drug dependence. To explore this claim, smokers were assessed for nicotine dependence, impulsivity, and capacity for goal-directed control over instrumental performance in an outcome devaluation procedure. Reduced goal-directed control was selectively associated with the Motor Impulsivity factor of Barretts Impulsivity Scale (BIS), which reflects propensity for action without thought. These data support the claim that human impulsivity is marked by impaired use of causal knowledge to make adaptive decisions. The predominance of habit learning may play a role in psychopathological conditions that are associated with trait impulsivity.