Lluís Fuentemilla

Go and no-go learning in reward and punishment: Interactions between affect and effect

Decision-making invokes two fundamental axes of control: affect or valence, spanning reward and punishment, and effect or action, spanning invigoration and inhibition. We studied the acquisition of instrumental responding in healthy human volunteers in a task in which we orthogonalized action requirements and outcome valence. Subjects were much more successful in learning active choices in rewarded conditions, and passive choices in punished conditions. Using computational reinforcement-learning models, we teased apart contributions from putatively instrumental and Pavlovian components in the generation of the observed asymmetry during learning. Moreover, using model-based fMRI, we showed that BOLD signals in striatum and substantia nigra/ventral tegmental area (SN/VTA) correlated with instrumentally learnt action values, but with opposite signs for go and no-go choices. Finally, we showed that successful instrumental learning depends on engagement of bilateral inferior frontal gyrus. Our behavioral and computational data showed that instrumental learning is contingent on overcoming inherent and plastic Pavlovian biases, while our neuronal data showed this learning is linked to unique patterns of brain activity in regions implicated in action and inhibition respectively.

Theta-Coupled Periodic Replay in Working Memory

Summary Working memory allows information from transient events to persist as active neural representations [1] that can be used for goal-directed behaviors such as decision making and learning [2, 3]. Computational modeling based on neuronal firing patterns in animals suggests that one putative mechanism enabling working memory is periodic reactivation (henceforth termed “replay”) of the maintained information coordinated by neural oscillations at theta (4–8 Hz) and gamma (30–80 Hz) frequency [4–6]. To investigate this possibility, we trained multivariate pattern classifier decoding algorithms on oscillatory brain responses to images depicting natural scenes, recorded with high temporal resolution via magnetoencephalography. These classifiers were applied to brain activity recorded during the subsequent five second maintenance of the scenes. This decoding revealed replay during the entire maintenance interval. Replay was specific to whether an indoor or an outdoor scene was maintained and whether maintenance centered on configural associations of scene elements or just single scene elements. Replay was coordinated by the phase of theta and the amount of theta coordination was correlated with working memory performance. By confirming the predictions of a mechanistic model and linking these to behavioral performance in humans, these findings identify theta-coupled replay as a mechanism of working memory maintenance.

Updating Fearful Memories with Extinction Training during Reconsolidation: A Human Study Using Auditory Aversive Stimuli

Learning to fear danger in the environment is essential to survival, but dysregulation of the fear system is at the core of many anxiety disorders. As a consequence, a great interest has emerged in developing strategies for suppressing fear memories in maladaptive cases. Recent research has focused in the process of reconsolidation where memories become labile after being retrieved. In a behavioral manipulation, Schiller et al., (2010) reported that extinction training, administrated during memory reconsolidation, could erase fear responses. The implications of this study are crucial for the possible treatment of anxiety disorders without the administration of drugs. However, attempts to replicate this effect by other groups have been so far unsuccessful. We sought out to reproduce Schiller et al., (2010) findings in a different fear conditioning paradigm based on auditory aversive stimuli instead of electric shock. Following a within-subject design, participants were conditioned to two different sounds and skin conductance response (SCR) was recorded as a measure of fear. Our results demonstrated that only the conditioned stimulus that was reminded 10 minutes before extinction training did not reinstate a fear response after a reminder trial consisting of the presentation of the unconditioned stimuli. For the first time, we replicated Schiller et al., (2010) behavioral manipulation and extended it to an auditory fear conditioning paradigm.

Dynamic Causal Models for phase coupling

This paper presents an extension of the Dynamic Causal Modelling (DCM) framework to the analysis of phase-coupled data. A weakly coupled oscillator approach is used to describe dynamic phase changes in a network of oscillators. The use of Bayesian model comparison allows one to infer the mechanisms underlying synchronization processes in the brain. For example, whether activity is driven by master-slave versus mutual entrainment mechanisms. Results are presented on synthetic data from physiological models and on MEG data from a study of visual working memory.

Human Hippocampus Arbitrates Approach-Avoidance Conflict

Summary Animal models of human anxiety often invoke a conflict between approach and avoidance [1, 2]. In these, a key behavioral assay comprises passive avoidance of potential threat and inhibition, both thought to be controlled by ventral hippocampus [2–6]. Efforts to translate these approaches to clinical contexts [7, 8] are hampered by the fact that it is not known whether humans manifest analogous approach-avoidance dispositions and, if so, whether they share a homologous neurobiological substrate [9]. Here, we developed a paradigm to investigate the role of human hippocampus in arbitrating an approach-avoidance conflict under varying levels of potential threat. Across four experiments, subjects showed analogous behavior by adapting both passive avoidance behavior and behavioral inhibition to threat level. Using functional magnetic resonance imaging (fMRI), we observe that threat level engages the anterior hippocampus, the human homolog of rodent ventral hippocampus [10]. Testing patients with selective hippocampal lesions, we demonstrate a causal role for the hippocampus with patients showing reduced passive avoidance behavior and inhibition across all threat levels. Our data provide the first human assay for approach-avoidance conflict akin to that of animal anxiety models. The findings bridge rodent and human research on passive avoidance and behavioral inhibition and furnish a framework for addressing the neuronal underpinnings of human anxiety disorders, where our data indicate a major role for the hippocampus.

Hippocampus-Dependent Strengthening of Targeted Memories via Reactivation during Sleep in Humans

Recent accumulating evidence in animals and humans has shown that memory strengthening occurs, at least partially, during sleep and relies on the covert reactivation of individual memory episodes. However, it remains to be determined whether the hippocampus critically promotes memory consolidation via the reactivation of individual memories during sleep. To investigate the hippocampal-dependent nature of this phenomenon in humans, we selected two groups of chronic temporal lobe epileptic (TLE) patients with selective unilateral (TLE+UHS) or bilateral (TLE+BHS) hippocampal sclerosis and a group of matched healthy controls, and we requested them to learn the association of sounds cueing the appearance of words. On the basis of other similar behavioral paradigms in healthy populations, sounds that cued only half of the learned memories were presented again during the slow-wave sleep stage (SWS) at night, thus promoting memory reactivation of a select set of encoded episodes. A memory test administered on the subsequent day showed that the strengthening of reactivated memories was observed only in the control subjects and TLE+UHS patients. Importantly, the amount of memory strengthening was predicted by the volume of spared hippocampus. Thus, the greater the structural integrity of the hippocampus, the higher the degree of memory benefit driven by memory reactivation. Finally, sleep-specific neurophysiological responses, such as spindles and slow waves, differed between the sample groups, and the spindle density during SWS predicted the degree of memory benefit observed on day 2. Taken together, these findings demonstrate that the hippocampus plays a crucial role in the consolidation of memories via covert reactivation during sleep.

Hippocampal Theta-Phase Modulation of Replay Correlates with Configural-Relational Short-Term Memory Performance

There is now growing evidence that the hippocampus generates theta rhythms that can phase bias fast neural oscillations in the neocortex, allowing coordination of widespread fast oscillatory populations outside limbic areas. A recent magnetoencephalographic study showed that maintenance of configural-relational scene information in a delayed match-to-sample (DMS) task was associated with replay of that information during the delay period. The periodicity of the replay was coordinated by the phase of the ongoing theta rhythm, and the degree of theta coordination during the delay period was positively correlated with DMS performance. Here, we reanalyzed these data to investigate which brain regions were involved in generating the theta oscillations that coordinated the periodic replay of configural–relational information. We used a beamformer algorithm to produce estimates of regional theta rhythms and constructed volumetric images of the phase-locking between the local theta cycle and the instances of replay (in the 13–80 Hz band). We found that individual differences in DMS performance for configural-relational associations were related to the degree of phase coupling of instances of cortical reactivations to theta oscillations generated in the right posterior hippocampus and the right inferior frontal gyrus. This demonstrates that the timing of memory reactivations in humans is biased toward hippocampal theta phase.

Reward Motivation Accelerates the Onset of Neural Novelty Signals in Humans to 85 Milliseconds

Summary The neural responses that distinguish novel from familiar items in recognition memory tasks are remarkably fast in both humans and nonhuman primates. In humans, the earliest onsets of neural novelty effects emerge at about ∼150–200 ms after stimulus onset [1–5]. However, in recognition memory studies with nonhuman primates, novelty effects can arise at as early as 70–80 ms [6, 7]. Here, we address the possibility that this large species difference in onset latencies is caused experimentally by the necessity of using reward reinforcement to motivate the detection of novel or familiar items in nonhuman primates but not in humans. Via magnetoencephalography in humans, we show in two experiments that the onset of neural novelty signals is accelerated from ∼200 ms to ∼85 ms if correct recognition memory for either novel or familiar items is rewarded. Importantly, this acceleration is independent of whether the detection of the novel or the familiar scenes is rewarded. Furthermore, this early novelty effect contributed to memory retrieval because neural reward responses, which were contingent upon novelty detection, followed ∼100 ms later. Thus, under the contextual influence of reward motivation, behaviorally relevant novelty signals emerge much faster than previously held possible in humans.

Theta oscillations orchestrate medial temporal lobe and neocortex in remembering autobiographical memories

Remembering autobiographical events can be associated with detailed visual imagery. The medial temporal lobe (MTL), precuneus and prefrontal cortex are held to jointly enable such vivid retrieval, but how these regions are orchestrated remains unclear. An influential prediction from animal physiology is that neural oscillations in theta frequency may be important. In this experiment, participants prospectively collected audio recordings describing personal autobiographical episodes or semantic knowledge over 2 to 7 months. These were replayed as memory retrieval cues while recording brain activity with magnetoencephalography (MEG). We identified a peak of theta power within a left MTL region of interest during both autobiographical and General Semantic retrieval. This MTL region was selectively phase-synchronized with theta oscillations in precuneus and medial prefrontal cortex, and this synchrony was higher during autobiographical as compared to General Semantic knowledge retrieval. Higher synchrony also predicted more detailed visual imagery during retrieval. Thus, theta phase-synchrony orchestrates in humans the MTL with a distributed neocortical memory network when vividly remembering autobiographical experiences.

Brain oscillatory activity associated with task switching and feedback processing

In this study, we sought to dissociate event-related potentials (ERPs) and the oscillatory activity associated with signals indicating feedback about performance (outcome-based behavioral adjustment) and the signals indicating the need to change or maintain a task set (rule-based behavioral adjustment). With this purpose in mind, we noninvasively recorded electroencephalographic signals, using a modified version of the Wisconsin card sorting task, in which feedback processing and task switching could be studied separately. A similar late positive component was observed for the switch and correct feedback signals on the first trials of a series, but feedback-related negativity was observed only for incorrect feedback. Moreover, whereas theta power showed a significant increase after a switch cue and after the first positive feedback of a new series, a selective frontal beta–gamma increase was observed exclusively in the first positive feedback (i.e., after the selection of the new rule). Importantly, for the switch cue, beta–alpha activity was suppressed rather than increased. This clear dissociation between the cue and feedback stimuli in task switching emphasizes the need to accurately study brain oscillatory activity to disentangle the role of different cognitive control processes.