Vincent Aubanel

The listening talker: A review of human and algorithmic context-induced modifications of speech

Speech output technology is finding widespread application, including in scenarios where intelligibility might be compromised - at least for some listeners - by adverse conditions. Unlike most current algorithms, talkers continually adapt their speech patterns as a response to the immediate context of spoken communication, where the type of interlocutor and the environment are the dominant situational factors influencing speech production. Observations of talker behaviour can motivate the design of more robust speech output algorithms. Starting with a listener-oriented categorisation of possible goals for speech modification, this review article summarises the extensive set of behavioural findings related to human speech modification, identifies which factors appear to be beneficial, and goes on to examine previous computational attempts to improve intelligibility in noise. The review concludes by tabulating 46 speech modifications, many of which have yet to be perceptually or algorithmically evaluated. Consequently, the review provides a roadmap for future work in improving the robustness of speech output.

Automatic recognition of regional phonological variation in conversational interaction

One key aspect of face-to-face communication concerns the differences that may exist between speakers native regional accents. This paper focuses on the characterization of regional phonological variation in a conversational setting. A new, interactive task was designed in which 12 pairs of participants engaged in a collaborative game leading them to produce a number of purpose-built names. In each game, the participants were native speakers of Southern French and Northern French, respectively. How the names were produced by each of the two participants was automatically determined from the recordings using ASR techniques and a pre-established set of possible regional variants along five phonological dimensions. A naive Bayes classifier was then applied to these phonetic forms, with a view to differentiating the speakers native regional accents. The results showed that native regional accent was correctly recognized for 79% of the speakers. These results also revealed or confirmed the existence of accent-dependent differences in how segments are phonetically realized, such as the affrication of /d/ in /di/ sequences. Our data allow us to better characterize the phonological and phonetic patterns associated with regional varieties of French on a large scale and in a natural, interactional situation.

Strategies adopted by talkers faced with fluctuating and competing-speech maskers

Studying how interlocutors exchange information efficiently during conversations in less-than-ideal acoustic conditions promises to both further the understanding of links between perception and production and inform the design of human-computer dialogue systems. The current study explored how interlocutors speech changes in the presence of fluctuating noise. Pairs of talkers were recorded while solving puzzles cooperatively in quiet and with modulated-noise or competing speech maskers whose silent intervals were manipulated to produce either temporally sparse or dense maskers. Talkers responded to masked conditions by both increasing the amount of speech produced and locally changing their speech activity patterns, resulting in a net reduction in the proportion of speech in temporal overlap with the maskers, with larger relative reductions for sparse maskers. An analysis of talker activity in the vicinity of masker onset and offset events showed a significant reduction in onsets following masker onsets, and a similar increase in onsets following masker offsets. These findings demonstrate that talkers are sensitive to masking noise and respond to its fluctuations by adopting a wait-and-talk strategy.

The Sharvard Corpus: a phonemically-balanced Spanish sentence resource for audiology.

Abstract Objective: The current study describes the collection of a new phonemically-balanced Spanish sentence resource, known as the Sharvard Corpus. Design: The resource contains 700 sentences inspired by the original English Harvard sentences along with speech recordings from a male and female native peninsular Spanish talker. Sentences each contain five keywords for scoring and are grouped into 70 lists of 10 sentences using an automatic phoneme-balancing procedure. Study sample: Twenty-three native Spanish listeners identified keywords in the Sharvard sentences in speech-shaped noise. Results: Psychometric functions for the Sharvard sentences indicate mean speech reception thresholds of − 6.07 and − 6.24 dB, and slopes of 10.53 and 11.03 percentage points per dB at the 50% keywords correct point for male and female talkers respectively. Conclusions: The resulting open source collection of Spanish sentence material for speech perception testing is available online.