Vanja Kovic

The Shape of Words in the Brain.

The principle of arbitrariness in language assumes that there is no intrinsic relationship between linguistic signs and their referents. However, a growing body of sound-symbolism research suggests the existence of some naturally-biased mappings between phonological properties of labels and perceptual properties of their referents (Maurer, Pathman, & Mondloch, 2006). We present new behavioural and neurophysiological evidence for the psychological reality of sound-symbolism. In a categorisation task that captures the processes involved in natural language interpretation, participants were faster to identify novel objects when label-object mappings were sound-symbolic than when they were not. Moreover, early negative EEG-waveforms indicated a sensitivity to sound-symbolic label-object associations (within 200ms of object presentation), highlighting the non-arbitrary relation between the objects and the labels used to name them. This sensitivity to sound-symbolic label-object associations may reflect a more general process of auditory-visual feature integration where properties of auditory stimuli facilitate a mapping to specific visual features.

Eye-tracking study of inanimate objects

Unlike the animate objects, where participants were consistent in their looking patterns, for inanimates it was difficult to identify both consistent areas of fixations and a consistent order of fixations. Furthermore, in comparison to animate objects, inanimates received significantly shorter total looking time, shorter longest looks and a smaller number of overall fixations. However, as with animates, looking patterns did not systematically differ between the naming and non-naming conditions. These results suggested that animacy, but not labelling, impacts on looking behaviour in this paradigm. In the light of feature-based accounts of semantic memory organization, one could interpret these findings as suggesting that processing of the animate objects is based on the saliency/diagnosticity of their visual features (which is then reflected through participants eye-movements towards those features), whereas processing of the inanimate objects is based more on functional features (which cannot be easily captured by looking behaviour in such a paradigm).

Crossmodal correspondences in natural language: Distribution of phonemes and consonant-vowel patterns in Serbian words denoting round and angular objects

Since described by Kohler more than half a century ago, phonetic–iconic correspondences have been demonstrated in a series of studies showing remarkable consistency in matches of pseudowords containing specific types of phonemes (e.g., Maluma or Takete) with rounded and angular shapes. If the effect found in these experiments reveals something about processes involved in natural language interpretation, we should expect similar association between phonological properties of objects’ labels and their perceptual properties to exist in natural language as well. However, results of the studies testing this effect in natural language are rather inconsistent and sometimes even contradictory. The aim of the present study was to test whether the distribution of phonemes and consonant-vowel patterns, previously found in pseudowords participants produced for the abstract visual patterns (Jankovic and Markovic, 2000, Perception 29 ECVP Abstract Supplement), could be found in the words of natural language. For 1066 nouns denoting round and angular shapes extracted from the Corpus of Serbian Language, distribution of phonemes and consonant-vowel patterns were analyzed. Results showed that words of Serbian language denoting sharp and rounded objects show similar patterns of phoneme and consonant-vowel distributions as those found in pseudowords produced for sharp and rounded visual stimuli, and therefore provide further evidence for cross-modal correspondences in natural language. These findings were discussed in the light of the role crossmodal correspondences can have in the natural language acquisition.

Now you see it, now you don’t: Design dependent sound symbolism effect in categorization studies

A number of studies have demonstrated sound-symbolism effects in adults and in children. Moreover, recently, ERP studies have shown that the sensitivity to sound-symbolic label–object associations occurs within 200 ms of object presentation (Kovic et al., 2010). It was argued that this effect may reflect a more general process of auditory–visual feature integration where properties of auditory stimuli facilitate a mapping to specific visual features. Here we demonstrate that the sound-symbolism effect is design dependent, namely — it occurs only when mapping from auditory to visual stimuli and not vice verse. Two groups of participants were recruited for solving the categorization task. They were presented them with 12 visual stimuli, half of which were rounded and another half of angular shapes. One group was trained to classify the rounded objects as ‘takete’ and the rounded ones as ‘maluma’, whereas the other group mapped ‘takete’ to rounded and ‘maluma’ to angular shapes. Moreover, half of these two groups heard the label before seeing the objects, whereas the other half was given the label after perceiving the object. The results revealed the sound-symbolism effect only in the group which was trained on the auditory–visual objects mapping and not in the one trained on the visual–auditory mappings. Thus, despite the previous findings we demonstrate that the sound-symbolism effect is not significant per se, but design-dependent and we argue that the sound brings up a mental image that is more constrained than the sounds brought up by a picture.

Measurement and processing of event-related brain potential records

The paper presents a system for measurement and processing of event-related brain potential records. ERP (event-related potentials) are described from point of view of its appearance and its significance in areas of cognitive neurosciences and neurophysiology. The system hardware is of modular structure, enabling easier changes of its elements. In the current version, software for stimulation and data processing implements oddball paradigm, with special attention to management of time intervals.

Timing in ERP measurement software

Measurement of event related potentials have significant role as a diagnostic method, as well as a method for researching cognitive functions of living beings. An important aspect of this measurement is timing. In this paper, one concept of structuring time intervals during the measurement is presented. Further, the implementation of this concept in measurement software is described, emphasizing algorithm in which stimuli events markers are being received directly from the software unit which controls starts of each stimuli presentation.

Software for biomedical P300 potential acquisition

The paper presents a software for P300 potential acquisition integrated into system for measurement, acquisition and processing of event-related brain potential records. ERP (event-related potentials) are analyzed from point of view of its appearance and its significance in areas of cognitive neurosciences and neurophysiology. The software implements a maximum potential value detection method of detecting P300 component, but also enables user to mark the point of P300 component in averaged waveform. At first place, the presented software is focused on deviant and standard stimuli management, and accurate time management, because of specific requirements of the research project it is made for. The software is currently localized in Serbian language.