Paolo Toffanin

Orienting attention to objects in visual short-term memory

We measured electroencephalographic activity during visual search of a target object among objects available to perception or among objects held in visual short-term memory (VSTM). For perceptual search, a single shape was shown first (pre-cue) followed by a search-array, and the task was to decide whether the pre-cue was or was not in the search-array. For search of VSTM, a search-array was shown first followed by a single shape (post-cue), and the task was to decide whether the post-cue was or was not in the previously displayed search-array. We focused on early lateralized electrical brain activity over posterior and temporal areas time-locked to search-arrays in pre-cue trials and to post-cues in post-cue trials. In Experiment 1, search-arrays were composed of two lateralized shapes, displayed in the upper/lower two quadrants of the monitor. In Experiment 2, search-arrays were composed of four shapes, displayed at the corners of an imaginary square centered on fixation. In pre-cue trials, we observed an N2pc of about equal amplitude and latency for search-arrays composed of two or four shapes. In post-cue trials, we observed N2pc-like activity with search-arrays composed of two shapes, that was however substantially attenuated with search-arrays composed of four shapes. For many aspects, attending to a perceptual object was functionally and neurally analogous to attending to an object held in VSTM, suggesting that spatial selective attention biases search of objects during both ongoing perception and retention.

Adaptive control of event integration

Identifying 2 target stimuli in a rapid stream of visual symbols is much easier if the 2nd target appears immediately after the 1st target (i.e., at Lag 1) than if distractor stimuli intervene. As this phenomenon comes with a strong tendency to confuse the order of the targets, it seems to be due to the integration of both targets into the same attentional episode or object file. The authors investigated the degree to which people can control the temporal extension of their (episodic) integration windows by manipulating the expectations participants had with regard to the time available for target processing. As predicted, expecting more time to process increased the number of order confusions at Lag 1. This was true for between-subjects and within-subjects (trial-to-trial) manipulations, suggesting that integration windows can be adapted actively and rather quickly.

Dynamic crossmodal links revealed by steady-state responses in auditory-visual divided attention.

Frequency tagging has been often used to study intramodal attention but not intermodal attention. We used EEG and simultaneous frequency tagging of auditory and visual sources to study intermodal focused and divided attention in detection and discrimination performance. Divided-attention costs were smaller, but still significant, in detection than in discrimination. The auditory steady-state response (SSR) showed no effects of attention at frontocentral locations, but did so at occipital locations where it was evident only when attention was divided between audition and vision. Similarly, the visual SSR at occipital locations was substantially enhanced when attention was divided across modalities. Both effects were equally present in detection and discrimination. We suggest that both effects reflect a common cause: An attention-dependent influence of auditory information processing on early cortical stages of visual information processing, mediated by enhanced effective connectivity between the two modalities under conditions of divided attention.

Rethinking neural efficiency: Effects of controlling for strategy use

A sentence verification task (SVT) was used to test whether differences in neural activation patterns that have been attributed to IQ may actually depend on differential strategy use between IQ groups. Electroencephalograms were recorded from 14 low (89 < IQ < 110) and 14 high (121 < IQ < 142) IQ individuals as they performed the SVT with either a spatial or verbal strategy. Event-related desynchronization in upper alpha (9.5-12.5 Hz) and theta (4-6 Hz) bands showed that different strategies evoked different activation patterns, but these patterns did not differ between groups. However, an IQ-related correlate was found in the preparation interval. Thus, although processing patterns during task performance seem to depend on the strategy used for task execution, preparation for task processing may depend on IQ.

The P4pc: an electrophysiological marker of attentional disengagement?

The processing of successive targets requires that attention be engaged and disengaged. Whereas attentional engagement can be studied by means of the N2pc component of the event-related potential (ERP), no ERP component has been linked to attentional disengagement. Here, we report the finding of such a component using an RSVP paradigm with multiple, successive targets and with a spatial-cuing paradigm. In both experiments, disengagement of attention was necessary to attend to subsequent targets. A distinct waveform following the N2pc, which we call the P4pc (Positivity 400 ms post-target posterior contralateral), was found. The P4pc was found when a lateralized cue indicated that attention would be needed for the processing of a target at either the same or a different location as the cue, but not when only the cue was to be responded to, indicating that the need to disengage attention is a prerequisite for the P4pc to occur. We expect the P4pc to provide a valuable addition to the set of electrophysiological measures used to study the dynamics and mechanisms of visual attention and visual search.

The Timing and Effort of Lexical Access in Natural and Degraded Speech

Understanding speech is effortless in ideal situations, and although adverse conditions, such as caused by hearing impairment, often render it an effortful task, they do not necessarily suspend speech comprehension. A prime example of this is speech perception by cochlear implant users, whose hearing prostheses transmit speech as a significantly degraded signal. It is yet unknown how mechanisms of speech processing deal with such degraded signals, and whether they are affected by effortful processing of speech. This paper compares the automatic process of lexical competition between natural and degraded speech, and combines gaze fixations, which capture the course of lexical disambiguation, with pupillometry, which quantifies the mental effort involved in processing speech. Listeners’ ocular responses were recorded during disambiguation of lexical embeddings with matching and mismatching durational cues. Durational cues were selected due to their substantial role in listeners’ quick limitation of the number of lexical candidates for lexical access in natural speech. Results showed that lexical competition increased mental effort in processing natural stimuli in particular in presence of mismatching cues. Signal degradation reduced listeners’ ability to quickly integrate durational cues in lexical selection, and delayed and prolonged lexical competition. The effort of processing degraded speech was increased overall, and because it had its sources at the pre-lexical level this effect can be attributed to listening to degraded speech rather than to lexical disambiguation. In sum, the course of lexical competition was largely comparable for natural and degraded speech, but showed crucial shifts in timing, and different sources of increased mental effort. We argue that well-timed progress of information from sensory to pre-lexical and lexical stages of processing, which is the result of perceptual adaptation during speech development, is the reason why in ideal situations speech is perceived as an undemanding task. Degradation of the signal or the receiver channel can quickly bring this well-adjusted timing out of balance and lead to increase in mental effort. Incomplete and effortful processing at the early pre-lexical stages has its consequences on lexical processing as it adds uncertainty to the forming and revising of lexical hypotheses.

Attentional Resources and Control

How we select, modulate and keep focused on information that is relevant to behaviour is critical to understanding human performance. Such diverse processes as memory storage and retrieval, action selection and decision-making cannot be fully described without consideration of the role that attention plays in them. On the one hand, attention is involved in the selection and modulation of incoming sensory information or information from memory. In this sense, attention determines the fate of selected items. Items that receive attention are processed more quickly and remembered better than items that do not receive attention’s ‘boost’ (Levin & Simons, 1997). On the other hand, because competition for cognitive resources is an integral part of most activities, attention is needed to maintain goal-directed behaviour. The role of attention in modulating sensory processes to select locations or objects in space was discussed in Chapter 3. In this chapter, we consider the investment of attentional resources across time and different tasks.

The Working Brain

Neuroergonomics has been defined as the study of the brain and behaviour at work (Parasuraman & Rizzo, 2007). The major goal of this new field is to use existing and emerging knowledge from the neurosciences to inform understanding of human behaviour and performance in work-relevant tasks. As such knowledge is gained, the hope is that we can design systems and work environments that are safe, efficient and enjoyable for their users. Reaching such a goal is made even more important by the relentless march of new, small information technologies in the marketplace—iPhones, global positioning systems (GPS), voice-operated devices and so forth impose new information-processing demands on their users. These devices can impair safety if they are used by people while they are simultaneously engaged in other activities, such as driving or walking across a busy intersection (e.g. Strayer & Drews, 2004).