Jaana Simola

Memory traces for words as revealed by the Mismatch Negativity

Brain responses to the same spoken syllable completing a Finnish word or a pseudo-word were studied. Native Finnish-speaking subjects were instructed to ignore the sound stimuli and watch a silent movie while the mismatch negativity (MMN), an automatic index of experience-dependent auditory memory traces, was recorded. The MMN to each syllable was larger when it completed a word than when it completed a pseudo-word. This enhancement, reaching its maximum amplitude at about 150 ms after the words recognition point, did not occur in foreign subjects who did not know any Finnish. These results provide the first demonstration of the presence of memory traces for individual spoken words in the human brain. Using whole-head magnetoencephalography, the major intracranial source of this word-related MMN was found in the left superior temporal lobe.

Preattentive extraction of abstract feature conjunctions from auditory stimulation as reflected by the mismatch negativity (MMN).

Brain mechanisms extracting invariant information from varying auditory inputs were studied using the mismatch-negativity (MMN) brain response. We wished to determine whether the preattentive sound-analysis mechanisms, reflected by MMN, are capable of extracting invariant relationships based on abstract conjunctions between two sound features. The standard stimuli varied over a large range in frequency and intensity dimensions following the rule that the higher the frequency, the louder the intensity. The occasional deviant stimuli violated this frequency-intensity relationship and elicited an MMN. The results demonstrate that preattentive processing of auditory stimuli extends to unexpectedly complex relationships between the stimulus features.

Combining eye movements and collaborative filtering for proactive information retrieval

We study a new task, proactive information retrieval by combining implicit relevance feedback and collaborative filtering. We have constructed a controlled experimental setting, a prototype application, in which the users try to find interesting scientific articles by browsing their titles. Implicit feedback is inferred from eye movement signals, with discriminative hidden Markov models estimated from existing data in which explicit relevance feedback is available. Collaborative filtering is carried out using the User Rating Profile model, a state-of-the-art probabilistic latent variable model, computed using Markov Chain Monte Carlo techniques. For new document titles the prediction accuracy with eye movements, collaborative filtering, and their combination was significantly better than by chance. The best prediction accuracy still leaves room for improvement but shows that proactive information retrieval and combination of many sources of relevance feedback is feasible.

The Impact of Salient Advertisements on Reading and Attention on Web Pages

Human vision is sensitive to salient features such as motion. Therefore, animation and onset of advertisements on Websites may attract visual attention and disrupt reading. We conducted three eye tracking experiments with authentic Web pages to assess whether (a) ads are efficiently ignored, (b) ads attract overt visual attention and disrupt reading, or (c) ads are covertly attended with distraction showing up indirectly in the reading performance. The Web pages contained an ad above a central text and another ad to the right of the text. In Experiments 1, 2, and 3A the task was to read for comprehension. Experiment 1 examined whether the degree of animation affects attention toward the ads. The results showed that ads were overtly attended during reading and that the dwell times on ads were the longest when the ad above was static and the other ad was animated. In Experiments 2 and 3, the ads appeared abruptly after a random time interval. The results showed that attention (i.e., the time when the eyes first entered an ad) was related to the ad onset time. This happened especially for the ad to the right, indicating that ads appearing close to the text region capture overt attention. In Experiment 3B the participants browsed the Web pages according to their own interest. The study demonstrated that salient ads attract overt visual attention and disrupt reading, but during free browsing, ads were viewed more frequently and for longer time than during reading.

Affective processing in natural scene viewing: valence and arousal interactions in eye-fixation-related potentials.

Attention is drawn to emotionally salient stimuli. The present study investigates processing of emotionally salient regions during free viewing of emotional scenes that were categorized according to the two-dimensional model comprising of valence (unpleasant, pleasant) and arousal (high, low). Recent studies have reported interactions between these dimensions, indicative of stimulus-evoked approach or withdrawal tendencies. We addressed the valence and arousal effects when emotional items were embedded in complex real-world scenes by analyzing both eye movement behavior and eye-fixation-related potentials (EFRPs) time-locked to the critical event of fixating the emotionally salient items for the first time. Both data sets showed an interaction between the valence and arousal dimensions. First, the fixation rates and gaze durations on emotionally salient regions were enhanced for unpleasant versus pleasant images in the high arousal condition. In the low arousal condition, both measures were enhanced for pleasant versus unpleasant images. Second, the EFRP results at 140-170 ms [P2] over the central site showed stronger responses for high versus low arousing images in the unpleasant condition. In addition, the parietal LPP responses at 400-500 ms post-fixation were enhanced for stimuli reflecting congruent stimulus dimensions, that is, stronger responses for high versus low arousing images in the unpleasant condition and stronger responses for low versus high arousing images in the pleasant condition. The present findings support the interactive two-dimensional approach, according to which the integration of valence and arousal recruits brain regions associated with action tendencies of approach or withdrawal.

Eye movement related brain responses to emotional scenes during free viewing

Emotional stimuli are preferentially processed over neutral stimuli. Previous studies, however, disagree on whether emotional stimuli capture attention preattentively or whether the processing advantage is dependent on allocation of attention. The present study investigated attention and emotion processes by measuring brain responses related to eye movement events while 11 participants viewed images selected from the International Affective Picture System (IAPS). Brain responses to emotional stimuli were compared between serial and parallel presentation. An “emotional” set included one image with high positive or negative valence among neutral images. A “neutral” set comprised four neutral images. The participants were asked to indicate which picture—if any—was emotional and to rate that picture on valence and arousal. In the serial condition, the event-related potentials (ERPs) were time-locked to the stimulus onset. In the parallel condition, the ERPs were time-locked to the first eye entry on an image. The eye movement results showed facilitated processing of emotional, especially unpleasant information. The EEG results in both presentation conditions showed that the LPP (“late positive potential”) amplitudes at 400–500 ms were enlarged for the unpleasant and pleasant pictures as compared to neutral pictures. Moreover, the unpleasant scenes elicited stronger responses than pleasant scenes. The ERP results did not support parafoveal emotional processing, although the eye movement results suggested faster attention capture by emotional stimuli. Our findings, thus, suggested that emotional processing depends on overt attentional resources engaged in the processing of emotional content. The results also indicate that brain responses to emotional images can be analyzed time-locked to eye movement events, although the response amplitudes were larger during serial presentation.

Topography of attention in the primary visual cortex

Previous research suggests that feedback circuits mediate the effect of attention to the primary visual cortex (V1). This inference is mainly based on temporal information of the responses, where late modulation is associated with feedback signals. However, temporal data alone are inconclusive because the anatomical hierarchy between cortical areas differs significantly from the temporal sequence of activation. In the current work, we relied on recent physiological and computational models of V1 network architecture, which have shown that the thalamic feedforward, local horizontal and feedback contribution are reflected in the spatial spread of responses. We used multifocal functional localizer and quantitative analysis in functional magnetic resonance imaging to determine the spatial scales of attention and sensory responses. Representations of 60 visual field regions in V1 were functionally localized and four of these regions were targets in a subsequent attention experiment, where human volunteers fixated centrally and performed a visual discrimination task at the attended location. Attention enhanced the peak amplitudes significantly more in the lower than in the upper visual field. This enhancement by attention spread with a 2.4 times larger radius (approximately 10 mm, assuming an average magnification factor) compared with the unattended response. The corresponding target region of interest was on average 20% stronger than that caused by the afferent sensory stimulation alone. This modulation could not be attributed to eye movements. Given the contemporary view of primate V1 connections, the activation spread along the cortex provides further evidence that the signal enhancement by spatial attention is dependent on feedback circuits.

Time dilates after spontaneous blinking

Accumulating evidence from pharmacology, neuroimaging, and genetics indicates that striatal dopamine influences time perception [1-5]. Despite these converging results, it is not known whether endogenous variations in dopamine underlie transient fluctuations in our perception of time. Here, we exploited the finding that striatal dopamine release is associated with an increase in spontaneous eye blink rate [6-8] to examine the relationship between intra-individual fluctuations in dopamine and interval timing. In two studies, participants overestimated visual subsecond and suprasecond and auditory subsecond intervals if they had blinked on the previous trial. These results are consistent with the hypothesis that transient fluctuations in striatal dopamine contribute to intra-individual variability in time perception.

Critical dynamics of endogenous fluctuations predict cognitive flexibility in the Go/NoGo task

Fluctuations with power-law scaling and long-range temporal correlations (LRTCs) are characteristic to human psychophysical performance. Systems operating in a critical state exhibit such LRTCs, but phenomenologically similar fluctuations and LRTCs may also be caused by slow decay of the system’s memory without the system being critical. Theoretically, criticality endows the system with the greatest representational capacity and flexibility in state transitions. Without criticality, however, slowly decaying system memory would predict inflexibility. We addressed these contrasting predictions of the ‘criticality’ and ‘long-memory’ candidate mechanisms of human behavioral LRTCs by using a Go/NoGo task wherein the commission errors constitute a measure of cognitive flexibility. Response time (RT) fluctuations in this task exhibited power-law frequency scaling, autocorrelations, and LRTCs. We show here that the LRTC scaling exponents, quantifying the strength of long-range correlations, were negatively correlated with the commission error rates. Strong LRTCs hence parallel optimal cognitive flexibility and, in line with the criticality hypothesis, indicate a functionally advantageous state. This conclusion was corroborated by a positive correlation between the LRTC scaling exponents and executive functions measured with the Rey-Osterrieth Complex Figure test. Our results hence support the notion that LRTCs arise from critical dynamics that is functionally significant for human cognitive performance.