Mike E. Le Pelley

When goals conflict with values: Counterproductive attentional and oculomotor capture by reward-related stimuli

Attention provides the gateway to cognition, by selecting certain stimuli for further analysis. Recent research demonstrates that whether a stimulus captures attention is not determined solely by its physical properties, but is malleable, being influenced by our previous experience of rewards obtained by attending to that stimulus. Here we show that this influence of reward learning on attention extends to task-irrelevant stimuli. In a visual search task, certain stimuli signaled the magnitude of available reward, but reward delivery was not contingent on responding to those stimuli. Indeed, any attentional capture by these critical distractor stimuli led to a reduction in the reward obtained. Nevertheless, distractors signaling large reward produced greater attentional and oculomotor capture than those signaling small reward. This counterproductive capture by task-irrelevant stimuli is important because it demonstrates how external reward structures can produce patterns of behavior that conflict with task demands, and similar processes may underlie problematic behavior directed toward real-world rewards.

Attention to Irrelevant Cues Is Related to Positive Symptoms in Schizophrenia

Many modern learning theories assume that the amount of attention to a cue depends on how well that cue predicted important events in the past. Schizophrenia is associated with deficits in attention and recent theories of psychosis have argued that positive symptoms such as delusions and hallucinations are related to a failure of selective attention. However, evidence demonstrating that attention to irrelevant cues is related to positive symptoms in schizophrenia is lacking. We used a novel method of measuring attention to nonpredictive (and thus irrelevant) cues in a causal learning test (Le Pelley ME, McLaren IP. Learned associability and associative change in human causal learning. Q J Exp Psychol B. 2003;56:68–79) to assess whether healthy adults and people with schizophrenia discriminate previously predictive and nonpredictive cues. In a series of experiments with independent samples, we demonstrated: (1) when people with schizophrenia who had severe positive symptoms successfully distinguished between predictive and nonpredictive cues during training, they failed to discriminate between predictive and nonpredictive cues relative to healthy adults during subsequent testing and (2) learning about nonpredictive cues was correlated with more severe positive symptoms scores in schizophrenia. These results suggest that positive symptoms of schizophrenia are related to increased attention to nonpredictive cues during causal learning. This deficit in selective attention results in learning irrelevant causal associations and may be the basis of positive symptoms in schizophrenia.

Stereotype formation: biased by association.

We propose that biases in attitude and stereotype formation might arise as a result of learned differences in the extent to which social groups have previously been predictive of behavioral or physical properties. Experiments 1 and 2 demonstrate that differences in the experienced predictiveness of groups with respect to evaluatively neutral information influence the extent to which participants later form attitudes and stereotypes about those groups. In contrast, Experiment 3 shows no influence of predictiveness when using a procedure designed to emphasize the use of higher level reasoning processes, a finding consistent with the idea that the root of the predictiveness bias is not in reasoning. Experiments 4 and 5 demonstrate that the predictiveness bias in formation of group beliefs does not depend on participants making global evaluations of groups. These results are discussed in relation to the associative mechanisms proposed by Mackintosh (1975) to explain similar phenomena in animal conditioning and associative learning.

Cognitive control and counterproductive oculomotor capture by reward-related stimuli

Two experiments investigated the extent to which value-modulated oculomotor capture is subject to top-down control. In these experiments, participants were never required to look at the reward-related stimuli; indeed, doing so was directly counterproductive because it caused omission of the reward that would otherwise have been obtained. In Experiment 1, participants were explicitly informed of this omission contingency. Nevertheless, they still showed counterproductive oculomotor capture by reward-related stimuli, suggesting that this effect is relatively immune to cognitive control. Experiment 2 more directly tested whether this capture is controllable by comparing the performance of participants who either had or had not been explicitly informed of the omission contingency. There was no evidence that value-modulated oculomotor capture differed between the two conditions, providing further evidence that this effect proceeds independently of cognitive control. Taken together, the results of the present research provide strong evidence for the automaticity and cognitive impenetrability of value-modulated attentional capture.

Learned Predictiveness Influences Rapid Attentional Capture: Evidence From the Dot Probe Task

Attentional theories of associative learning and categorization propose that learning about the predictiveness of a stimulus influences the amount of attention that is paid to that stimulus. Three experiments tested this idea by looking at the extent to which stimuli that had previously been experienced as predictive or nonpredictive in a categorization task were able to capture attention in a dot probe task. Consistent with certain attentional theories of learning, responses to the dot probe were faster when it appeared in a location cued by a predictive stimulus compared to a location cued by a nonpredictive stimulus. This result was obtained only with short (250-ms or 350-ms) but not long (1,000-ms) delays between onset of the stimuli and the dot probe, suggesting that the observed spatial cuing effect reflects the operation of a relatively rapid, automatic process. These findings are consistent with the approach to the relationship between attention and learning taken by the class of models exemplified by Mackintoshs (1975) theory.

Oculomotor capture by stimuli that signal the availability of reward

It is well known that eye movement patterns are influenced by both goal- and salience-driven factors. Recent studies, however, have demonstrated that objects that are nonsalient and task irrelevant can still capture our eyes if moving our eyes to those objects has previously produced reward. Here we demonstrate that training such an association between eye movements to an object and delivery of reward is not needed. Instead, an object that merely signals the availability of reward captures the eyes even when it is physically nonsalient and never relevant for the task. Furthermore, we show that oculomotor capture by reward is more reliably observed in saccades with short latencies. We conclude that a stimulus signaling high reward has the ability to capture the eyes independently of bottom-up physical salience or top-down task relevance and that the effect of reward affects early selection processes.

Uncertainty and predictiveness determine attention to cues during human associative learning

Prior research has suggested that attention is determined by exploiting what is known about the most valid predictors of outcomes and exploring those stimuli that are associated with the greatest degree of uncertainty about subsequent events. Previous studies of human contingency learning have revealed evidence for one or other of these processes, but differences in the designs and procedures of these studies make it difficult to pinpoint the crucial determinant of whether attentional exploitation or exploration will dominate. Here we present two studies in which we systematically manipulated both the predictiveness of cues and uncertainty regarding the outcomes with which they were associated. This allowed us to demonstrate, for the first time, evidence of both attentional exploration and exploitation within the same experiment. Moreover, while the effect of predictiveness persisted to influence the rate of novel learning about the same cues in a second stage, the effect of uncertainty did not. This suggests that attentional exploration is more sensitive to a change of context than is exploitation. The pattern of data is simulated with a hybrid attentional model.

The positive symptoms of acute schizophrenia and latent inhibition in humans and animals: Underpinned by the same process(es)?

Introduction. It has been suggested that the positive symptoms of acute schizophrenia are a consequence of a disruption of the process that produces latent inhibition (slower acquisition of conditioned responding after preexposure to the conditioned stimulus) and that this effect can be modelled by pro- and antipsychotic compounds in healthy participants and in nonhuman animals. This idea assumes that latent inhibition in humans and animals is underpinned by the same process(es). Method. First, we question the equivalence of human and animal latent inhibition. Second, we review the studies that have examined latent inhibition in populations with schizophrenia and in healthy populations after administration of amphetamine or haloperidol. Results. Theoretical analysis of the similarities and differences in latent inhibition effects, and the procedures used to generate them, in humans and animals renders the suggested equivalence between them unconvincing. The studies examining latent inhibition in populations with schizophrenia and in healthy populations after administration of amphetamine or haloperidol are marked by a number of methodological shortcomings and reveal discrepant results. Conclusions. The theoretical and empirical analyses provide little support for a common process underlying deficits of latent inhibition in patients exhibiting positive symptoms of acute schizophrenia, and such deficits in experimental models in healthy humans and infrahumans.

Goal-Directed and Habit-Like Modulations of Stimulus Processing during Reinforcement Learning

Recent research has shown that perceptual processing of stimuli previously associated with high-value rewards is automatically prioritized even when rewards are no longer available. It has been hypothesized that such reward-related modulation of stimulus salience is conceptually similar to an “attentional habit.” Recording event-related potentials in humans during a reinforcement learning task, we show strong evidence in favor of this hypothesis. Resistance to outcome devaluation (the defining feature of a habit) was shown by the stimulus-locked P1 component, reflecting activity in the extrastriate visual cortex. Analysis at longer latencies revealed a positive component (corresponding to the P3b, from 550–700 ms) sensitive to outcome devaluation. Therefore, distinct spatiotemporal patterns of brain activity were observed corresponding to habitual and goal-directed processes. These results demonstrate that reinforcement learning engages both attentional habits and goal-directed processes in parallel. Consequences for brain and computational models of reinforcement learning are discussed. SIGNIFICANCE STATEMENT The human attentional network adapts to detect stimuli that predict important rewards. A recent hypothesis suggests that the visual cortex automatically prioritizes reward-related stimuli, driven by cached representations of reward value; that is, stimulus–response habits. Alternatively, the neural system may track the current value of the predicted outcome. Our results demonstrate for the first time that visual cortex activity is increased for reward-related stimuli even when the rewarding event is temporarily devalued. In contrast, longer-latency brain activity was specifically sensitive to transient changes in reward value. Therefore, we show that both habit-like attention and goal-directed processes occur in the same learning episode at different latencies. This result has important consequences for computational models of reinforcement learning.

Goal-directed EEG activity evoked by discriminative stimuli in reinforcement learning

In reinforcement learning (RL), discriminative stimuli (S) allow agents to anticipate the value of a future outcome, and the response that will produce that outcome. We examined this processing by recording EEG locked to S during RL. Incentive value of outcomes and predictive value of S were manipulated, allowing us to discriminate between outcome-related and response-related activity. S predicting the correct response differed from nonpredictive S in the P2. S paired with high-value outcomes differed from those paired with low-value outcomes in a frontocentral positivity and in the P3b. A slow negativity then distinguished between predictive and nonpredictive S. These results suggest that, first, attention prioritizes detection of informative S. Activation of mental representations of these informative S then retrieves representations of outcomes, which in turn retrieve representations of responses that previously produced those outcomes.