Tamás M. Bőhm

Modelling the Emergence and Dynamics of Perceptual Organisation in Auditory Streaming

Many sound sources can only be recognised from the pattern of sounds they emit, and not from the individual sound events that make up their emission sequences. Auditory scene analysis addresses the difficult task of interpreting the sound world in terms of an unknown number of discrete sound sources (causes) with possibly overlapping signals, and therefore of associating each event with the appropriate source. There are potentially many different ways in which incoming events can be assigned to different causes, which means that the auditory system has to choose between them. This problem has been studied for many years using the auditory streaming paradigm, and recently it has become apparent that instead of making one fixed perceptual decision, given sufficient time, auditory perception switches back and forth between the alternatives—a phenomenon known as perceptual bi- or multi-stability. We propose a new model of auditory scene analysis at the core of which is a process that seeks to discover predictable patterns in the ongoing sound sequence. Representations of predictable fragments are created on the fly, and are maintained, strengthened or weakened on the basis of their predictive success, and conflict with other representations. Auditory perceptual organisation emerges spontaneously from the nature of the competition between these representations. We present detailed comparisons between the model simulations and data from an auditory streaming experiment, and show that the model accounts for many important findings, including: the emergence of, and switching between, alternative organisations; the influence of stimulus parameters on perceptual dominance, switching rate and perceptual phase durations; and the build-up of auditory streaming. The principal contribution of the model is to show that a two-stage process of pattern discovery and competition between incompatible patterns can account for both the contents (perceptual organisations) and the dynamics of human perception in auditory streaming.

Characterising switching behaviour in perceptual multi-stability

When people experience an unchanging sensory input for a long period of time, their perception tends to switch stochastically and unavoidably between alternative interpretations of the sensation; a phenomenon known as perceptual bi-stability or multi-stability. The huge variability in the experimental data obtained in such paradigms makes it difficult to distinguish typical patterns of behaviour, or to identify differences between switching patterns. Here we propose a new approach to characterising switching behaviour based upon the extraction of transition matrices from the data, which provide a compact representation that is well-understood mathematically. On the basis of this representation we can characterise patterns of perceptual switching, visualise and simulate typical switching patterns, and calculate the likelihood of observing a particular switching pattern. The proposed method can support comparisons between different observers, experimental conditions and even experiments. We demonstrate the insights offered by this approach using examples from our experiments investigating multi-stability in auditory streaming. However, the methodology is generic and thus widely applicable in studies of multi-stability in any domain.

Event-related potential correlates of sound organization: early sensory and late cognitive effects.

We tested whether incoming sounds are processed differently depending on how the preceding sound sequence has been interpreted by the brain. Sequences of a regularly repeating three-tone pattern, the perceived organization of which spontaneously switched back and forth between two alternative interpretations, were delivered to listeners. Occasionally, a regular tone was exchanged for a slightly or moderately lower one (deviants). The electroencephalogram (EEG) was recorded while listeners continuously marked their perception of the sound sequence. We found that for both the regular and the deviant tones, the early exogenous P1 and N1 amplitudes varied together with the perceived sound organization. Percept-dependent effects on the late endogenous N2 and P3a amplitudes were only found for deviant tones. These results suggest that the perceived sound organization affects sound processing both by modulating what information is extracted from incoming sounds as well as by influencing how deviant sound events are evaluated for further processing.

Automatic classification of regular vs. irregular phonation types

Irregular phonation (also called creaky voice, glottalization and laryngealization) may have various communicative functions in speech. Thus the automatic classification of phonation type into regular and irregular can have a number of applications in speech technology. In this paper, we propose such a classifier that extracts six acoustic cues from vowels and then labels them as regular or irregular by means of a support vector machine. We integrated cues from earlier phonation type classifiers and improved their performance in five out of the six cases. The classifier with the improved cue set produced a 98.85% hit rate and a 3.47% false alarm rate on a subset of the TIMIT corpus.

Transforming modal voice into irregular voice by amplitude scaling of individual glottal cycles

Irregular phonation can serve as a cue to segmental contrasts and prosodic structure as well as to the affective state and identity of the speaker. Thus algorithms for transforming between voice qualities, such as regular and irregular phonation, may contribute to building more natural sounding, expressive and personalized speech synthesizers. We describe a semiautomatic transformation method that introduces irregular pitch periods into a modal speech signal by amplitude scaling of the individual cycles. First the periods are separated by windowing, then multiplied by scaling factors, and finally overlapped and added. Thus, amplitude irregularities are introduced via boosting or attenuating selected cycles. The abrupt, substantial changes in cycle lengths that are characteristic of naturally‐occurring irregular phonation can be achieved by removing (scaling to zero) one or more consecutive periods. A freely available graphical tool has been developed for copying stylized pulse patterns (glottal pulse spac...

Do audio-visual motion cues promote segregation of auditory streams?

An audio-visual experiment using moving sound sources was designed to investigate whether the analysis of auditory scenes is modulated by synchronous presentation of visual information. Listeners were presented with an alternating sequence of two pure tones delivered by two separate sound sources. In different conditions, the two sound sources were either stationary or moving on random trajectories around the listener. Both the sounds and the movement trajectories were derived from recordings in which two humans were moving with loudspeakers attached to their heads. Visualized movement trajectories modeled by a computer animation were presented together with the sounds. In the main experiment, behavioral reports on sound organization were collected from young healthy volunteers. The proportion and stability of the different sound organizations were compared between the conditions in which the visualized trajectories matched the movement of the sound sources and when the two were independent of each other. The results corroborate earlier findings that separation of sound sources in space promotes segregation. However, no additional effect of auditory movement per se on the perceptual organization of sounds was obtained. Surprisingly, the presentation of movement-congruent visual cues did not strengthen the effects of spatial separation on segregating auditory streams. Our findings are consistent with the view that bistability in the auditory modality can occur independently from other modalities.