Oren Griffiths

When goals conflict with values: Counterproductive attentional and oculomotor capture by reward-related stimuli

Attention provides the gateway to cognition, by selecting certain stimuli for further analysis. Recent research demonstrates that whether a stimulus captures attention is not determined solely by its physical properties, but is malleable, being influenced by our previous experience of rewards obtained by attending to that stimulus. Here we show that this influence of reward learning on attention extends to task-irrelevant stimuli. In a visual search task, certain stimuli signaled the magnitude of available reward, but reward delivery was not contingent on responding to those stimuli. Indeed, any attentional capture by these critical distractor stimuli led to a reduction in the reward obtained. Nevertheless, distractors signaling large reward produced greater attentional and oculomotor capture than those signaling small reward. This counterproductive capture by task-irrelevant stimuli is important because it demonstrates how external reward structures can produce patterns of behavior that conflict with task demands, and similar processes may underlie problematic behavior directed toward real-world rewards.

Attention to Irrelevant Cues Is Related to Positive Symptoms in Schizophrenia

Many modern learning theories assume that the amount of attention to a cue depends on how well that cue predicted important events in the past. Schizophrenia is associated with deficits in attention and recent theories of psychosis have argued that positive symptoms such as delusions and hallucinations are related to a failure of selective attention. However, evidence demonstrating that attention to irrelevant cues is related to positive symptoms in schizophrenia is lacking. We used a novel method of measuring attention to nonpredictive (and thus irrelevant) cues in a causal learning test (Le Pelley ME, McLaren IP. Learned associability and associative change in human causal learning. Q J Exp Psychol B. 2003;56:68–79) to assess whether healthy adults and people with schizophrenia discriminate previously predictive and nonpredictive cues. In a series of experiments with independent samples, we demonstrated: (1) when people with schizophrenia who had severe positive symptoms successfully distinguished between predictive and nonpredictive cues during training, they failed to discriminate between predictive and nonpredictive cues relative to healthy adults during subsequent testing and (2) learning about nonpredictive cues was correlated with more severe positive symptoms scores in schizophrenia. These results suggest that positive symptoms of schizophrenia are related to increased attention to nonpredictive cues during causal learning. This deficit in selective attention results in learning irrelevant causal associations and may be the basis of positive symptoms in schizophrenia.

Overt attention and predictiveness in human contingency learning.

Two experiments used eye-tracking procedures to investigate the relationship between attention and associative learning in human participants. These experiments found greater overt attention to cues experienced as predictive of the outcomes with which they were paired, than to cues experienced as nonpredictive. Moreover, this attentional bias persisted into a second training phase when all cues were equally predictive of the outcomes with which they were paired, and it was accompanied by a related bias in the rate of learning about these cues. These findings are consistent with the attentional model of associative learning proposed by Mackintosh (1975), but not with that proposed by Pearce and Hall (1980).

Selective attention in human associative learning and recognition memory.

Four experiments examined the role of selective attention in a new causal judgment task that allowed measurement of both causal strength and cue recognition. In Experiments 1 and 2, blocking was observed; pretraining with 1 cue (A) resulted in reduced learning about a 2nd cue (B) when those 2 cues were trained in compound (AB+). Participants also demonstrated decreased recognition performance for the causally redundant Cue B, suggesting that less attention had been paid to it in training. This is consistent with the idea that attention is preferentially allocated toward the more predictive Cue A, and away from the less predictive Cue B (e.g., N. J. Mackintosh, 1975). Contrary to this hypothesis, in Experiments 3 and 4, participants demonstrated poorer recognition for the most predictive cues, relative to control cues. A new model, which is based on N. J. Mackintoshs (1975) model, is proposed to account for the observed relationship between the extent to which each cue is attended to, learned about, and later recognized

Negative priming reduces affective ratings

The mere exposure effect is the commonly observed increase in pleasantness ratings of stimuli that have been given prior exposure. According to the fluency attribution account of the mere exposure effect, repeated presentations of a stimulus lead to increased ease of processing, which in turn is attributed to pleasantness. If so, processing fluency manipulated by means other than repetition should influence liking. In the present experiment, processing fluency was manipulated using a negative priming procedure, and its influence on affective judgement was examined. Previously ignored stimuli were responded to slower (negative priming) and were rated as less pleasant than controls. It was concluded that decreased processing fluency decreases liking of previously ignored stimuli.

Attentional mechanisms in learned predictiveness.

Cues that reliably predict an outcome in an initial phase of training (Phase 1) are learned faster in a second phase of training (Phase 2) than cues that were unreliable in Phase 1. This result is observed despite objectively equal relationships between the cues and the outcomes in Phase 2, and consequently constitutes a nonnormative bias in learning. The present experiments sought to confirm that this learned predictiveness effect is the product of attentional processes (Experiment 1), and to test further whether these processes are under voluntary control or are automatic in nature (Experiment 2). In addition to the usual outcome prediction measure, eye-gaze behavior was also monitored. The results indicated an important role for top-down strategic attentional processes in the learned predictiveness task. In contrast, no evidence for an automatic attentional bias was found.

Negative Transfer in Human Associative Learning

Models of attentional allocation in associative learning are typically structured according to one of two guiding principles: the predictiveness principle, which posits that attention is paid to cues that have reliably predicted an outcome in the past, or the uncertainty principle, which states that attention is paid to cues about which little is known. Both principles are well supported by studies of animals. However, in studies of human learning, there is very little direct empirical support for the uncertainty principle. In the study reported here, we addressed this gap by investigating negative transfer, a phenomenon that may provide unique support for the uncertainty principle. In two human learning experiments using an allergist task, we replicated the primary findings of previous research on animal learning. We believe that these data provide the first direct evidence for the uncertainty principle in human associative learning.

Attentional changes in blocking are not a consequence of lateral inhibition

In three human causal learning experiments, we examined attentional modulation in the blocking task, in which participants typically learn little about a novel cue B when it is paired with a previously trained, predictive cue A. Evidence indicates that this blocking training led to a decrement in attention to the blocked cue B. The present experiments addressed whether this decrease in attention to the blocked cue could be better explained as being due to lateral inhibition from the pretrained cue A to the blocked cue B, or as a cue-specific property that is not conditional on the presence or absence of other stimuli. Strong effects of learned predictiveness were observed on participants’ causal judgments (Experiment 1) and choice behavior (Experiments 2 and 3). However, no evidence for lateral inhibitory processes emerged in any of the experiments, despite explicit attempts to maximize experimental sensitivity to this effect. The results are discussed in the context of formal models of the operation of attentional processes in human and animal learning.

Feature-based versus category-based induction with uncertain categories.

Previous research has suggested that when feature inferences have to be made about an instance whose category membership is uncertain, feature-based inductive reasoning is used to the exclusion of category-based induction. These results contrast with the observation that people can and do use category-based induction when category membership is known. The present experiments examined the conditions that drive feature-based and category-based strategies in induction under category uncertainty. Specifically, 2 experiments investigated whether reliance on feature-based inductive strategies is a product of the lack of coherence in the categories used in previous research or is due to the use of a decision-only induction procedure. Experiment 1 found that feature-based reasoning remained the preferred strategy even when categories with relatively high internal coherence were used. Experiment 2 found a shift toward category-based reasoning when participants were trained to classify category members prior to feature induction. Together, these results suggest that an appropriate conceptual representation must be formed through experience with a category before it is likely to be used as a basis for feature induction.

Goal-Directed and Habit-Like Modulations of Stimulus Processing during Reinforcement Learning

Recent research has shown that perceptual processing of stimuli previously associated with high-value rewards is automatically prioritized even when rewards are no longer available. It has been hypothesized that such reward-related modulation of stimulus salience is conceptually similar to an “attentional habit.” Recording event-related potentials in humans during a reinforcement learning task, we show strong evidence in favor of this hypothesis. Resistance to outcome devaluation (the defining feature of a habit) was shown by the stimulus-locked P1 component, reflecting activity in the extrastriate visual cortex. Analysis at longer latencies revealed a positive component (corresponding to the P3b, from 550–700 ms) sensitive to outcome devaluation. Therefore, distinct spatiotemporal patterns of brain activity were observed corresponding to habitual and goal-directed processes. These results demonstrate that reinforcement learning engages both attentional habits and goal-directed processes in parallel. Consequences for brain and computational models of reinforcement learning are discussed. SIGNIFICANCE STATEMENT The human attentional network adapts to detect stimuli that predict important rewards. A recent hypothesis suggests that the visual cortex automatically prioritizes reward-related stimuli, driven by cached representations of reward value; that is, stimulus–response habits. Alternatively, the neural system may track the current value of the predicted outcome. Our results demonstrate for the first time that visual cortex activity is increased for reward-related stimuli even when the rewarding event is temporarily devalued. In contrast, longer-latency brain activity was specifically sensitive to transient changes in reward value. Therefore, we show that both habit-like attention and goal-directed processes occur in the same learning episode at different latencies. This result has important consequences for computational models of reinforcement learning.