Miriam Sebold

Model-Based and Model-Free Decisions in Alcohol Dependence

Background: Human and animal work suggests a shift from goal-directed to habitual decision-making in addiction. However, the evidence for this in human alcohol dependence is as yet inconclusive. Methods: Twenty-six healthy controls and 26 recently detoxified alcohol-dependent patients underwent behavioral testing with a 2-step task designed to disentangle goal-directed and habitual response patterns. Results: Alcohol-dependent patients showed less evidence of goal-directed choices than healthy controls, particularly after losses. There was no difference in the strength of the habitual component. The group differences did not survive controlling for performance on the Digit Symbol Substitution Task. Conclusion: Chronic alcohol use appears to selectively impair goal-directed function, rather than promoting habitual responding. It appears to do so particularly after nonrewards, and this may be mediated by the effects of alcohol on more general cognitive functions subserved by the prefrontal cortex.

Pavlovian-to-instrumental transfer effects in the nucleus accumbens relate to relapse in alcohol dependence

In detoxified alcohol‐dependent patients, alcohol‐related stimuli can promote relapse. However, to date, the mechanisms by which contextual stimuli promote relapse have not been elucidated in detail. One hypothesis is that such contextual stimuli directly stimulate the motivation to drink via associated brain regions like the ventral striatum and thus promote alcohol seeking, intake and relapse. Pavlovian‐to‐Instrumental‐Transfer (PIT) may be one of those behavioral phenomena contributing to relapse, capturing how Pavlovian conditioned (contextual) cues determine instrumental behavior (e.g. alcohol seeking and intake). We used a PIT paradigm during functional magnetic resonance imaging to examine the effects of classically conditioned Pavlovian stimuli on instrumental choices in n = 31 detoxified patients diagnosed with alcohol dependence and n = 24 healthy controls matched for age and gender. Patients were followed up over a period of 3 months. We observed that (1) there was a significant behavioral PIT effect for all participants, which was significantly more pronounced in alcohol‐dependent patients; (2) PIT was significantly associated with blood oxygen level‐dependent (BOLD) signals in the nucleus accumbens (NAcc) in subsequent relapsers only; and (3) PIT‐related NAcc activation was associated with, and predictive of, critical outcomes (amount of alcohol intake and relapse during a 3 months follow‐up period) in alcohol‐dependent patients. These observations show for the first time that PIT‐related BOLD signals, as a measure of the influence of Pavlovian cues on instrumental behavior, predict alcohol intake and relapse in alcohol dependence.

Default mode network subsystem alterations in obsessive-compulsive disorder

BACKGROUND Although neurobiological models of obsessive-compulsive disorder (OCD) traditionally emphasise the central role of corticostriatal brain regions, studies of default mode network integrity have garnered increasing interest, but have produced conflicting results. AIMS To resolve these discrepant findings by examining the integrity of default mode network subsystems in OCD. METHOD Comparison of seed-based resting-state functional connectivity of 11 default mode network components between 46 patients with OCD and 46 controls using functional magnetic resonance imaging. RESULTS Significantly reduced connectivity within the dorsal medial prefrontal cortex self subsystem was identified in the OCD group, and remained significant after controlling for medication status and life-time history of affective disorders. Further, greater connectivity between the self subsystem and salience and attention networks was observed. CONCLUSIONS Results indicate that people with OCD show abnormalities in a neural system previously associated with self-referential processing in healthy individuals, and suggest the need for examination of dynamic interactions between this default mode network subsystem and other large-scale networks in this disorder.

Processing speed enhances model-based over model-free reinforcement learning in the presence of high working memory functioning

Theories of decision-making and its neural substrates have long assumed the existence of two distinct and competing valuation systems, variously described as goal-directed vs. habitual, or, more recently and based on statistical arguments, as model-free vs. model-based reinforcement-learning. Though both have been shown to control choices, the cognitive abilities associated with these systems are under ongoing investigation. Here we examine the link to cognitive abilities, and find that individual differences in processing speed covary with a shift from model-free to model-based choice control in the presence of above-average working memory function. This suggests shared cognitive and neural processes; provides a bridge between literatures on intelligence and valuation; and may guide the development of process models of different valuation components. Furthermore, it provides a rationale for individual differences in the tendency to deploy valuation systems, which may be important for understanding the manifold neuropsychiatric diseases associated with malfunctions of valuation.

Pavlovian-to-Instrumental Transfer in Alcohol Dependence: A Pilot Study

Background: Pavlovian processes are thought to play an important role in the development, maintenance and relapse of alcohol dependence, possibly by influencing and usurping ongoing thought and behavior. The influence of pavlovian stimuli on ongoing behavior is paradigmatically measured by pavlovian-to-instrumental transfer (PIT) tasks. These involve multiple stages and are complex. Whether increased PIT is involved in human alcohol dependence is uncertain. We therefore aimed to establish and validate a modified PIT paradigm that would be robust, consistent and tolerated by healthy controls as well as by patients suffering from alcohol dependence, and to explore whether alcohol dependence is associated with enhanced PIT. Methods: Thirty-two recently detoxified alcohol-dependent patients and 32 age- and gender-matched healthy controls performed a PIT task with instrumental go/no-go approach behaviors. The task involved both pavlovian stimuli associated with monetary rewards and losses, and images of drinks. Results: Both patients and healthy controls showed a robust and temporally stable PIT effect. Strengths of PIT effects to drug-related and monetary conditioned stimuli were highly correlated. Patients more frequently showed a PIT effect, and the effect was stronger in response to aversively conditioned CSs (conditioned suppression), but there was no group difference in response to appetitive CSs. Conclusion: The implementation of PIT has favorably robust properties in chronic alcohol-dependent patients and in healthy controls. It shows internal consistency between monetary and drug-related cues. The findings support an association of alcohol dependence with an increased propensity towards PIT. 2014 S. Karger AG, Basel

The PhysIO toolbox for modeling physiological noise in fMRI data

BACKGROUND Physiological noise is one of the major confounds for fMRI. A common class of correction methods model noise from peripheral measures, such as ECGs or pneumatic belts. However, physiological noise correction has not emerged as a standard preprocessing step for fMRI data yet due to: (1) the varying data quality of physiological recordings, (2) non-standardized peripheral data formats and (3) the lack of full automatization of processing and modeling physiology, required for large-cohort studies. NEW METHODS We introduce the PhysIO Toolbox for preprocessing of physiological recordings and model-based noise correction. It implements a variety of noise models, such as RETROICOR, respiratory volume per time and heart rate variability responses (RVT/HRV). The toolbox covers all intermediate steps - from flexible read-in of data formats to GLM regressor/contrast creation - without any manual intervention. RESULTS We demonstrate the workflow of the toolbox and its functionality for datasets from different vendors, recording devices, field strengths and subject populations. Automatization of physiological noise correction and performance evaluation are reported in a group study (N=35). COMPARISON WITH EXISTING METHODS The PhysIO Toolbox reproduces physiological noise patterns and correction efficacy of previously implemented noise models. It increases modeling robustness by outperforming vendor-provided peak detection methods for physiological cycles. Finally, the toolbox offers an integrated framework with full automatization, including performance monitoring, and flexibility with respect to the input data. CONCLUSIONS Through its platform-independent Matlab implementation, open-source distribution, and modular structure, the PhysIO Toolbox renders physiological noise correction an accessible preprocessing step for fMRI data.

When habits are dangerous - Alcohol expectancies and habitual decision-making predict relapse in alcohol dependence

BACKGROUND Addiction is supposedly characterized by a shift from goal-directed to habitual decision making, thus facilitating automatic drug intake. The two-step task allows distinguishing between these mechanisms by computationally modeling goal-directed and habitual behavior as model-based and model-free control. In addicted patients, decision making may also strongly depend upon drug-associated expectations. Therefore, we investigated model-based versus model-free decision making and its neural correlates as well as alcohol expectancies in alcohol-dependent patients and healthy controls and assessed treatment outcome in patients. METHODS Ninety detoxified, medication-free, alcohol-dependent patients and 96 age- and gender-matched control subjects underwent functional magnetic resonance imaging during the two-step task. Alcohol expectancies were measured with the Alcohol Expectancy Questionnaire. Over a follow-up period of 48 weeks, 37 patients remained abstinent and 53 patients relapsed as indicated by the Alcohol Timeline Followback method. RESULTS Patients who relapsed displayed reduced medial prefrontal cortex activation during model-based decision making. Furthermore, high alcohol expectancies were associated with low model-based control in relapsers, while the opposite was observed in abstainers and healthy control subjects. However, reduced model-based control per se was not associated with subsequent relapse. CONCLUSIONS These findings suggest that poor treatment outcome in alcohol dependence does not simply result from a shift from model-based to model-free control but is instead dependent on the interaction between high drug expectancies and low model-based decision making. Reduced model-based medial prefrontal cortex signatures in those who relapse point to a neural correlate of relapse risk. These observations suggest that therapeutic interventions should target subjective alcohol expectancies.

Too Difficult to Stop: Mechanisms Facilitating Relapse in Alcohol Dependence

Background: In alcohol and other substance dependencies, patients often suffer relapse despite better knowledge and their intention to remain abstinent. A variety of neurotransmitter systems and their respective alterations due to the chronic drug intake are involved in mechanisms that facilitate relapse. It has been postulated that these neurotransmitter systems are related to changes in motivational and learning mechanisms, and engender a shift from goal-directed to habitual behavior in dependent patients that facilitates drug-seeking behavior. Methods: We review learning mechanisms facilitating relapse, as identified and tested to date. We focus on studies examining the interaction between alcohol-related changes in monoaminergic neurotransmission and their respective effects on pavlovian and operant learning mechanisms in alcohol dependence. Results: Animal experiments and first human studies suggest that chronic alcohol intake impairs goal-directed behavior and facilitates habitual drug intake. Key symptoms of alcohol dependence such as tolerance development, withdrawal, craving and reduced control of alcohol intake can be explained by alcohol-induced alteration of dopaminergic neurotransmission and its GABAergic and glutamatergic modulation and their respective effects on pavlovian and operant conditioning as well as pavlovian-to-instrumental transfer. Conclusion: Chronic alcohol intake impairs neurotransmitter systems that regulate prefrontal-striatal circuits and interfere with goal-directed decision-making and the acquisition of new, non-drug-related behavior patterns. Alcohol craving induced by pavlovian conditioned cues can facilitate habitual drug intake. Such learning mechanisms and their alterations by chronic alcohol intake might be targeted by specific interventions.

No association of goal-directed and habitual control with alcohol consumption in young adults

Alcohol dependence is a mental disorder that has been associated with an imbalance in behavioral control favoring model‐free habitual over model‐based goal‐directed strategies. It is as yet unknown, however, whether such an imbalance reflects a predisposing vulnerability or results as a consequence of repeated and/or excessive alcohol exposure. We, therefore, examined the association of alcohol consumption with model‐based goal‐directed and model‐free habitual control in 188 18‐year‐old social drinkers in a two‐step sequential decision‐making task while undergoing functional magnetic resonance imaging before prolonged alcohol misuse could have led to severe neurobiological adaptations. Behaviorally, participants showed a mixture of model‐free and model‐based decision‐making as observed previously. Measures of impulsivity were positively related to alcohol consumption. In contrast, neither model‐free nor model‐based decision weights nor the trade‐off between them were associated with alcohol consumption. There were also no significant associations between alcohol consumption and neural correlates of model‐free or model‐based decision quantities in either ventral striatum or ventromedial prefrontal cortex. Exploratory whole‐brain functional magnetic resonance imaging analyses with a lenient threshold revealed early onset of drinking to be associated with an enhanced representation of model‐free reward prediction errors in the posterior putamen. These results suggest that an imbalance between model‐based goal‐directed and model‐free habitual control might rather not be a trait marker of alcohol intake per se.

Model-based control in dimensional psychiatry

We use parallel interacting goal-directed and habitual strategies to make our daily decisions. The arbitration between these strategies is relevant to inflexible repetitive behaviors in psychiatric disorders. Goal-directed control, also known as model-based control, is based on an affective outcome relying on a learned internal model to prospectively make decisions. In contrast, habit control, also known as model-free control, is based on an integration of previous reinforced learning autonomous of the current outcome value and is implicit and more efficient but at the cost of greater inflexibility. The concept of model-based control can be further extended into pavlovian processes. Here we describe and compare tasks that tap into these constructs and emphasize the clinical relevance and translation of these tasks in psychiatric disorders. Together, these findings highlight a role for model-based control as a transdiagnostic impairment underlying compulsive behaviors and representing a promising therapeutic target.