Maartje Schreuder

Prosodic Processes in Language and Music

In dit proefschrift vergelijken we taal en muziek. Zoals componist Lerdahl en taalkundige Jackendoff laten zien in hun boek ‘A Generative Theory of Tonal Music’, hebben deze twee cognitieve kanten van ons gedrag overeenkomsten, zoals hierarchische structuur, waarin prominente elementen worden onderscheiden van minder belangrijke elementen door middel van zogenaamde preferentieregels. Ook ritme en fraseringsverschijnselen hebben ze met elkaar gemeen. Op preferentieregels gebaseerde theorieen voor fonologie, zoals Optimaliteitstheorie, laten zien dat die overeenkomsten voor fonologische en muzikale analyses nog opvallender zijn dan Lerdahl en Jackendoff suggereren. We laten zien dat muziektheorie kan helpen om taalkundige kwesties op te lossen die moeilijk op te lossen zijn met de taalkundige theorie alleen. Drie van zulke kwesties onderzoeken we experimenteel. De eerste kwestie is of spraak slechts ingekort en gecomprimeerd wordt als mensen sneller gaan spreken, met de zelfde ritmische structuur, of dat het spraakritme verandert onder invloed van tempo. De tweede kwestie is de vraag of recursie gevonden kan worden in de fonologie. Worden fraseringskenmerken zoals accentverschuiving herhaaldelijk toegepast in ingebedde fonologische frases? De derde kwestie gaat over majeur- en mineurmodaliteit in de intonatie van vrolijke en sombere spraak. Een van de belangrijkste vindingen is dat luisteraars hun waarneming soms blijken te baseren op auditieve illusies en niet altijd op het werkelijke geluidsignaal. Luisteraars horen wat ze verwachten te horen. Net als in muziek wordt ritme als regelmatiger gehoord dan het in werkelijkheid is. De resultaten van dit onderzoek bevestigen de aanname dat taal en muziek overeenkomsten hebben op een aantal kenmerken. Beide bestaan ze uit geluid en beide soorten geluid worden door de luisteraar op een vergelijkbare manier gestructureerd. English: This dissertation makes a comparison of language and music. As composer Lerdahl and linguist Jackendoff show in their ‘Generative Theory of Tonal Music’, these two cognitive behaviors share aspects, such as hierarchical structure, in which prominent elements are separated from non-prominent elements by means of preference rules and rhythmic and phrasing phenomena. Recent constraint-based approaches to phonology, such as Optimality Theory, show that the similarities are even more striking for phonological and musical analyses. This dissertation shows that music theory may help to solve linguistic issues with which linguistic theory alone finds it hard to deal. Three such issues are investigated experimentally. The first issue is whether speech is just shortened and compressed when people speak faster, with the same rhythmic structure, or whether the speech rhythm changes. The second issue is the question whether recursion can be found in phonology. Are phrasing phenomena such as early accent placement applied repeatedly in embedded phonological phrases? The third issue is major and minor modality in intonation contours of cheerful and sad speech. One of the main findings is that listeners appear sometimes to base their perception on auditory illusions, not always on the sound signal as it is. Listeners hear what they expect to hear. As in music, rhythm is perceived as more regular than it is in reality. The results of this research confirm the assumption that speech and music share many features. Both are ‘made of’ sound, and both kinds of sound signal are structured by the listener in a similar way.

Masking the Identities of Celebrities and Personally Familiar Individuals: Effects on Visual and Auditory Recognition Performance

The current study compared the effectiveness of masking on recognition performance for faces and voices of celebrities and personally familiar individuals. On the basis of the theory suggesting stronger memory representations for personally familiar individuals, we expected masking to be more effective for celebrities than for personally familiar stimulus persons. Furthermore, we sought to replicate the face recognition advantage with masked stimuli. Face pixelation and electronic changes of the voice pitch were applied as masking techniques, using four masking levels for each stimulus. Thirty-one undergraduate students were presented with the masked faces and voices of 10 celebrities and 10 personally familiar fellow students. As expected, more correct recognitions occurred for faces than for voices, suggesting that masking does not counteract the mechanisms causing the face recognition advantage. Unexpectedly, masking effectiveness did not differ between celebrities and personally familiar individuals. This may be due to the type of personally familiar individuals used. Within personally familiar stimuli, increased familiarity did not predict the effectiveness of masking. Whereas the highest masking level eliminated speaker recognition, masking did not fully eliminate face recognition. From a practical perspective, the findings especially question the suitability of pixelation as a means for identity concealment.

Frightened by the perpetrator's voice: Startle responsivity and cognitive processing predict earwitness speaker identification

This study was inspired by the case of a robbery victim who was startled and reminded of the crime upon hearing a strangers voice, while not clearly recognizing the speaker. To investigate whether specific voices can modulate startle reactions and thereby predict speaker identification, we presented an audio hijack scenario to 84 participants and afterwards asked them to identify the perpetrator among neutral and negative speech fragments, while measuring flash-evoked eye-blink startle responses. Furthermore, we addressed data-driven cognitive processing during the audio scenario as a potential moderator in voice discrimination. Negative speech and the perpetrators voice led to potentiated startle. Enhanced startle was positively associated with voice discrimination, but only in neutral speech fragments. In negative fragments, this association was weakened as a function of self-reported levels of data-driven processing during encoding. Thus, startle responses can generally predict accurate voice recognition, but speech emotionality and cognitive processing moderate this relationship.