Viljami R. Salmela

Music and speech prosody: a common rhythm

Disorders of music and speech perception, known as amusia and aphasia, have traditionally been regarded as dissociated deficits based on studies of brain damaged patients. This has been taken as evidence that music and speech are perceived by largely separate and independent networks in the brain. However, recent studies of congenital amusia have broadened this view by showing that the deficit is associated with problems in perceiving speech prosody, especially intonation and emotional prosody. In the present study the association between the perception of music and speech prosody was investigated with healthy Finnish adults (n = 61) using an on-line music perception test including the Scale subtest of Montreal Battery of Evaluation of Amusia (MBEA) and Off-Beat and Out-of-key tasks as well as a prosodic verbal task that measures the perception of word stress. Regression analyses showed that there was a clear association between prosody perception and music perception, especially in the domain of rhythm perception. This association was evident after controlling for music education, age, pitch perception, visuospatial perception, and working memory. Pitch perception was significantly associated with music perception but not with prosody perception. The association between music perception and visuospatial perception (measured using analogous tasks) was less clear. Overall, the pattern of results indicates that there is a robust link between music and speech perception and that this link can be mediated by rhythmic cues (time and stress).

Media multitasking is associated with distractibility and increased prefrontal activity in adolescents and young adults

The current generation of young people indulges in more media multitasking behavior (e.g., instant messaging while watching videos) in their everyday lives than older generations. Concerns have been raised about how this might affect their attentional functioning, as previous studies have indicated that extensive media multitasking in everyday life may be associated with decreased attentional control. In the current study, 149 adolescents and young adults (aged 13-24years) performed speech-listening and reading tasks that required maintaining attention in the presence of distractor stimuli in the other modality or dividing attention between two concurrent tasks. Brain activity during task performance was measured using functional magnetic resonance imaging (fMRI). We studied the relationship between self-reported daily media multitasking (MMT), task performance and brain activity during task performance. The results showed that in the presence of distractor stimuli, a higher MMT score was associated with worse performance and increased brain activity in right prefrontal regions. The level of performance during divided attention did not depend on MMT. This suggests that daily media multitasking is associated with behavioral distractibility and increased recruitment of brain areas involved in attentional and inhibitory control, and that media multitasking in everyday life does not translate to performance benefits in multitasking in laboratory settings.

Brain activity during divided and selective attention to auditory and visual sentence comprehension tasks

Using functional magnetic resonance imaging (fMRI), we measured brain activity of human participants while they performed a sentence congruence judgment task in either the visual or auditory modality separately, or in both modalities simultaneously. Significant performance decrements were observed when attention was divided between the two modalities compared with when one modality was selectively attended. Compared with selective attention (i.e., single tasking), divided attention (i.e., dual-tasking) did not recruit additional cortical regions, but resulted in increased activity in medial and lateral frontal regions which were also activated by the component tasks when performed separately. Areas involved in semantic language processing were revealed predominantly in the left lateral prefrontal cortex by contrasting incongruent with congruent sentences. These areas also showed significant activity increases during divided attention in relation to selective attention. In the sensory cortices, no crossmodal inhibition was observed during divided attention when compared with selective attention to one modality. Our results suggest that the observed performance decrements during dual-tasking are due to interference of the two tasks because they utilize the same part of the cortex. Moreover, semantic dual-tasking did not appear to recruit additional brain areas in comparison with single tasking, and no crossmodal inhibition was observed during intermodal divided attention.

Working memory resources are shared across sensory modalities

A common assumption in the working memory literature is that the visual and auditory modalities have separate and independent memory stores. Recent evidence on visual working memory has suggested that resources are shared between representations, and that the precision of representations sets the limit for memory performance. We tested whether memory resources are also shared across sensory modalities. Memory precision for two visual (spatial frequency and orientation) and two auditory (pitch and tone duration) features was measured separately for each feature and for all possible feature combinations. Thus, only the memory load was varied, from one to four features, while keeping the stimuli similar. In Experiment 1, two gratings and two tones—both containing two varying features—were presented simultaneously. In Experiment 2, two gratings and two tones—each containing only one varying feature—were presented sequentially. The memory precision (delayed discrimination threshold) for a single feature was close to the perceptual threshold. However, as the number of features to be remembered was increased, the discrimination thresholds increased more than twofold. Importantly, the decrease in memory precision did not depend on the modality of the other feature(s), or on whether the features were in the same or in separate objects. Hence, simultaneously storing one visual and one auditory feature had an effect on memory precision equal to those of simultaneously storing two visual or two auditory features. The results show that working memory is limited by the precision of the stored representations, and that working memory can be described as a resource pool that is shared across modalities.

Spatial frequency tuning of brightness polarity identification

Recent studies have shown that cells in the primary visual cortex can, in addition to borders, also encode surface brightness. Whether the brightness is encoded by a large extraclassical receptive field or by a filling-in type mechanism activated by the luminance border is not known. These explanations imply different spatial frequency tunings for the underlying mechanism. In a psychophysical masking paradigm we measured spatial frequency tuning functions for identification of both luminance polarity (bright/dark) and luminance border orientation of oval and circular luminance patches with variable diameters (0.2-10 deg). For both tasks we obtained nearly overlapping narrow (1.5 octave) bandpass masking tuning functions centered at 1.5-5.0 c/deg. Stimulus size and shape had only minimal effect on the tuning functions. The results favor the idea of brightness filling-in and suggest that the cells activated by the luminance border modulate the activity of the cells signaling surface brightness. Further, the brightness processing mechanism is spatial frequency selective.

Gaming is related to enhanced working memory performance and task-related cortical activity

Gaming experience has been suggested to lead to performance enhancements in a wide variety of working memory tasks. Previous studies have, however, mostly focused on adult expert gamers and have not included measurements of both behavioral performance and brain activity. In the current study, 167 adolescents and young adults (aged 13-24 years) with different amounts of gaming experience performed an n-back working memory task with vowels, with the sensory modality of the vowel stream switching between audition and vision at random intervals. We studied the relationship between self-reported daily gaming activity, working memory (n-back) task performance and related brain activity measured using functional magnetic resonance imaging (fMRI). The results revealed that the extent of daily gaming activity was related to enhancements in both performance accuracy and speed during the most demanding (2-back) level of the working memory task. This improved working memory performance was accompanied by enhanced recruitment of a fronto-parietal cortical network, especially the dorsolateral prefrontal cortex. In contrast, during the less demanding (1-back) level of the task, gaming was associated with decreased activity in the same cortical regions. Our results suggest that a greater degree of daily gaming experience is associated with better working memory functioning and task difficulty-dependent modulation in fronto-parietal brain activity already in adolescence and even when non-expert gamers are studied. The direction of causality within this association cannot be inferred with certainty due to the correlational nature of the current study.

Low-level features determine brightness in White's and Benary's illusions

We masked Whites and Benarys brightness illusions and simultaneous contrast with narrowband visual noise and measured detection thresholds and brightness. The noise was either isotropic or orientation filtered. A narrow spatial frequency tuning was found for detection and brightness for every stimulus. A narrow orientation tuning was also found: the strength of the illusions decreased (White and Benary) or increased (White) depending on the orientation of the mask. The critical borders were always of the same contrast polarity. The results suggest that the brightness in figure-ground scenes is determined by mechanisms integrating incremental and decremental borders in early visual cortices.

Radial Frequency Analysis of Contour Shapes in the Visual Cortex

Cumulative psychophysical evidence suggests that the shape of closed contours is analysed by means of their radial frequency components (RFC). However, neurophysiological evidence for RFC-based representations is still missing. We investigated the representation of radial frequency in the human visual cortex with functional magnetic resonance imaging. We parametrically varied the radial frequency, amplitude and local curvature of contour shapes. The stimuli evoked clear responses across visual areas in the univariate analysis, but the response magnitude did not depend on radial frequency or local curvature. Searchlight-based, multivariate representational similarity analysis revealed RFC specific response patterns in areas V2d, V3d, V3AB, and IPS0. Interestingly, RFC-specific representations were not found in hV4 or LO, traditionally associated with visual shape analysis. The modulation amplitude of the shapes did not affect the responses in any visual area. Local curvature, SF-spectrum and contrast energy related representations were found across visual areas but without similar specificity for visual area that was found for RFC. The results suggest that the radial frequency of a closed contour is one of the cortical shape analysis dimensions, represented in the early and mid-level visual areas.

Brightness and transparency in the early visual cortex

Several psychophysical studies have shown that transparency can have drastic effects on brightness and lightness. However, the neural processes generating these effects have remained unresolved. Several lines of evidence suggest that the early visual cortex is important for brightness perception. While single cell recordings suggest that surface brightness is represented in the primary visual cortex, the results of functional magnetic resonance imaging (fMRI) studies have been discrepant. In addition, the location of the neural representation of transparency is not yet known. We investigated whether the fMRI responses in areas V1, V2, and V3 correlate with brightness and transparency. To dissociate the blood oxygen level-dependent (BOLD) response to brightness from the response to local border contrast and mean luminance, we used variants of Whites brightness illusion, both opaque and transparent, in which luminance increments and decrements cancel each other out. The stimuli consisted of a target surface and a surround. The surround luminance was always sinusoidally modulated at 0.5 Hz to induce brightness modulation to the target. The target luminance was constant or modulated in counterphase to null brightness modulation. The mean signal changes were calculated from the voxels in V1, V2, and V3 corresponding to the retinotopic location of the target surface. The BOLD responses were significantly stronger for modulating brightness than for stimuli with constant brightness. In addition, the responses were stronger for transparent than for opaque stimuli, but there was more individual variation. No interaction between brightness and transparency was found. The results show that the early visual areas V1-V3 are sensitive to surface brightness and transparency and suggest that brightness and transparency are represented separately.

Brain activity associated with selective attention, divided attention and distraction

Top-down controlled selective or divided attention to sounds and visual objects, as well as bottom-up triggered attention to auditory and visual distractors, has been widely investigated. However, no study has systematically compared brain activations related to all these types of attention. To this end, we used functional magnetic resonance imaging (fMRI) to measure brain activity in participants performing a tone pitch or a foveal grating orientation discrimination task, or both, distracted by novel sounds not sharing frequencies with the tones or by extrafoveal visual textures. To force focusing of attention to tones or gratings, or both, task difficulty was kept constantly high with an adaptive staircase method. A whole brain analysis of variance (ANOVA) revealed fronto-parietal attention networks for both selective auditory and visual attention. A subsequent conjunction analysis indicated partial overlaps of these networks. However, like some previous studies, the present results also suggest segregation of prefrontal areas involved in the control of auditory and visual attention. The ANOVA also suggested, and another conjunction analysis confirmed, an additional activity enhancement in the left middle frontal gyrus related to divided attention supporting the role of this area in top-down integration of dual task performance. Distractors expectedly disrupted task performance. However, contrary to our expectations, activations specifically related to the distractors were found only in the auditory and visual cortices. This suggests gating of the distractors from further processing perhaps due to strictly focused attention in the current demanding discrimination tasks.