Aaron M. Bornstein

Dissociating hippocampal and striatal contributions to sequential prediction learning

Behavior may be generated on the basis of many different kinds of learned contingencies. For instance, responses could be guided by the direct association between a stimulus and response, or by sequential stimulus–stimulus relationships (as in model‐based reinforcement learning or goal‐directed actions). However, the neural architecture underlying sequential predictive learning is not well understood, in part because it is difficult to isolate its effect on choice behavior. To track such learning more directly, we examined reaction times (RTs) in a probabilistic sequential picture identification task in healthy individuals. We used computational learning models to isolate trial‐by‐trial effects of two distinct learning processes in behavior, and used these as signatures to analyse the separate neural substrates of each process. RTs were best explained via the combination of two delta rule learning processes with different learning rates. To examine neural manifestations of these learning processes, we used functional magnetic resonance imaging to seek correlates of time‐series related to expectancy or surprise. We observed such correlates in two regions, hippocampus and striatum. By estimating the learning rates best explaining each signal, we verified that they were uniquely associated with one of the two distinct processes identified behaviorally. These differential correlates suggest that complementary anticipatory functions drive each region’s effect on behavior. Our results provide novel insights as to the quantitative computational distinctions between medial temporal and basal ganglia learning networks and enable experiments that exploit trial‐by‐trial measurement of the unique contributions of both hippocampus and striatum to response behavior.

Reminders of past choices bias decisions for reward in humans

We provide evidence that decisions are made by consulting memories for individual past experiences, and that this process can be biased in favour of past choices using incidental reminders. First, in a standard rewarded choice task, we show that a model that estimates value at decision-time using individual samples of past outcomes fits choices and decision-related neural activity better than a canonical incremental learning model. In a second experiment, we bias this sampling process by incidentally reminding participants of individual past decisions. The next decision after a reminder shows a strong influence of the action taken and value received on the reminded trial. These results provide new empirical support for a decision architecture that relies on samples of individual past choice episodes rather than incrementally averaged rewards in evaluating options and has suggestive implications for the underlying cognitive and neural mechanisms.

Reinstated episodic context guides sampling-based decisions for reward

How does experience inform decisions? In episodic sampling, decisions are guided by a few episodic memories of past choices. This process can yield choice patterns similar to model-free reinforcement learning; however, samples can vary from trial to trial, causing decisions to vary. Here we show that context retrieved during episodic sampling can cause choice behavior to deviate sharply from the predictions of reinforcement learning. Specifically, we show that, when a given memory is sampled, choices (in the present) are influenced by the properties of other decisions made in the same context as the sampled event. This effect is mediated by fMRI measures of context retrieval on each trial, suggesting a mechanism whereby cues trigger retrieval of context, which then triggers retrieval of other decisions from that context. This result establishes a new avenue by which experience can guide choice and, as such, has broad implications for the study of decisions.

What’s past is present: Reminders of past choices bias decisions for reward in humans

We provide evidence that decisions are made by consulting memories for individual past experiences, and that this process can be biased in favor of past choices using incidental reminders. First, in a standard rewarded choice task, we show that a model that estimates value at decision-time using individual samples of past outcomes fits choices and decision-related neural activity better than a canonical incremental learning model. In a second experiment, we bias this sampling process by incidentally reminding participants of individual past decisions. The next decision after a reminder shows a strong influence of the action taken and value received on the reminded trial. These results provide new empirical support for a decision architecture that relies on samples of individual past choice episodes rather than incrementally averaged rewards in evaluating options, and has suggestive implications for the underlying cognitive and neural mechanisms.

An unusual complication associated with blepharoplasty

The usual complications associated with blepharoplasty are well known. This report presents a case of second-degree burns of the face which occurred while oxygen was being administered to a patient undergoing blepharoplasty and a heat-producing cautery was being used for hemostasis. A possible cause for the complication is suggested. Surgeons are warned of the potential problem that may occur when this combination of factors are present.

Enhanced Motion Perception as a Psychophysical Marker for Autism

In recent decades, the incidence of autism has reached epidemic proportions. The ever-mounting burden of disease from autism spectrum disorders highlights the urgency of developing effective treatment options. However, this remains a formidable task. Although autism is characterized by core symptoms

Unblocking the Neural Substrates of Model-Based Value

A central premise of prominent reinforcement learning (RL) models is that actions and cues are considered valuable only insofar as justified by the rewards that actors learn to expect with them. This expectation can be adjusted with further experience, to the degree that new rewards received are

Refresh my memory: Episodic memory reinstatements intrude on working memory maintenance

A fundamental question in memory research is how different forms of memory interact. Previous research has shown that people rely on working memory (WM) in short-term recognition tasks; a common view is that episodic memory (EM) only influences performance on these tasks when WM maintenance is disrupted. However, retrieval of memories from EM has been widely observed during brief periods of quiescence, raising the possibility that EM retrievals during maintenance-critically, before a response can be prepared-might affect short-term recognition memory performance even in the absence of distraction. We hypothesized that this influence would be mediated by the lingering presence of reactivated EM content in WM. We obtained support for this hypothesis in three experiments, showing that delay-period EM reactivation introduces incidentally-associated information (context) into WM, and that these retrieved associations negatively impact subsequent recognition, leading to substitution errors (Experiment 1) and slowing of accurate responses (Experiment 2). fMRI pattern analysis showed that slowing is mediated by the content of EM reinstatement (Experiment 3). These results expose a previously hidden influence of EM on WM, raising new questions about the adaptive nature of their interaction.

Putting value in context: A role for context memory in decisions for reward

How does experience inform decisions? In episodic sampling, decisions are guided by a few episodic memories of past choices. This process can yield choice patterns similar to model-free Reinforcement Learning (RL); however, samples can vary from trial to trial, causing decisions to vary. Here, we show that context retrieved during episodic sampling can cause choice behavior to deviate sharply from the predictions of RL. Specifically, we show that, when a given memory is sampled, choices (in the present) are influenced by the properties of other decisions made in the same context as the sampled event. This effect is mediated by fMRI measures of context retrieval on each trial, suggesting a mechanism whereby cues trigger retrieval of context, which then triggers retrieval of other decisions from that context. This result establishes a new avenue by which experience can guide choice, and as such has broad implications for the study of decisions.How does experience inform decisions? In episodic sampling, decisions are guided by a few episodic memories of past choices. This process can yield choice patterns similar to model-free Reinforcement Learning (RL); however, samples can vary from trial to trial, causing decisions to vary. Here, we show that context retrieved during episodic sampling can cause choice behavior to deviate sharply from the predictions of RL. Specifically, we show that, when a given memory is sampled, choices (in the present) are influenced by the properties of other decisions made in the same context as the sampled event. This effect is mediated by fMRI measures of context retrieval on each trial, suggesting a mechanism whereby cues trigger retrieval of context, which then triggers retrieval of other decisions from that context. This result establishes a new avenue by which experience can guide choice, and as such has broad implications for the study of decisions.

Perceptual decisions result from the continuous accumulation of memory and sensory evidence

Expectations can inform fast, accurate decisions. But what informs expectations? Here we test the hypothesis that expectations are set by dynamic inference from memory. Participants performed a cue-guided perceptual decision task with independently-varying memory and sensory evidence. Cues established expectations by reminding participants of past stimulus-stimulus pairings, which predicted the likely target in a subsequent noisy image stream. Participant’s responses used both memory and sensory information, weighted by their relative reliability. Formal model comparison showed that the sensory evidence accumulation was best explained when its parameters were set dynamically at each trial by evidence accumulated from memory. Supporting this model, neural pattern analysis revealed that responses to the probe were modulated by the specific content and fidelity of memory reinstatement that occurred before the probe appeared. Together, these results suggest that perceptual decisions arise from the continuous and integrated accumulation of memory and sensory evidence.