Sara Candeias

Prosodic and Phonetic Features for Speaking Styles Classification and Detection

This study presents an approach to the task of automatically classifying and detecting speaking styles. The detection of speaking styles is useful for the segmentation of multimedia data into consistent parts and has important applications, such as identifying speech segments to train acoustic models for speech recognition. In this work the database consists of daily news broadcasts in Portuguese television, on which two main speaking styles are evident: read speech from voice-over and anchors, and spontaneous speech from interviews and commentaries. Using a combination of phonetic and prosodic features we can separate these two speaking styles with a good accuracy (93.7% read, 69.5% spontaneous). This is performed in two steps. The first step separates the speech segments from the non-speech audio segments and the second step classifies read versus spontaneous speaking style. The use of phonetic and prosodic features provides alternative information that leads to an improvement of the classification and detection task.

Generating a pronunciation dictionary for European Portuguese using a joint-sequence model with embedded stress assignment

This paper addresses the problem of grapheme to phoneme conversion to create a pronunciation dictionary from a vocabulary of the most frequent words in European Portuguese. A system based on a mixed approach funded on a stochastic model with embedded rules for stressed vowel assignment is described. The implemented model can generate pronunciations from unrestricted words; however, a dictionary with the 40k most frequent words was constructed and corrected interactively. The dictionary includes homographs with multiplepronunciations. The vocabulary was defined using the CETEMPúblico corpus. The model and dictionary are publicly available.

Read That Article: Exploring Synergies between Gaze and Speech Interaction

Gaze information has the potential to benefit Human-Computer Interaction (HCI) tasks, particularly when combined with speech. Gaze can improve our understanding of the user intention, as a secondary input modality, or it can be used as the main input modality by users with some level of permanent or temporary impairments. In this paper we describe a multimodal HCI system prototype which supports speech, gaze and the combination of both. The system has been developed for Active Assisted Living scenarios.

Characterizing Parkinson’s Disease Speech by Acoustic and Phonetic Features

This study intends to identify acoustic and phonetic characteristics of the speech of Parkinson’s Disease (PD) patients, usually manifesting hypokinetic dysarthria. A speech database has been collected from a control group and from a group of patients with similar PD severity, but with different degrees of hypokinetic dysarthria. First and second formant frequencies of vowels in continuous speech were analyzed. Several classifiers were built using phonetic features and a range of acoustic features based on cepstral coefficients with the objective of identifying hypokinetic dysarthria. Results show a centralization of vowel formant frequencies for PD speech, as expected. However, some of the features highlighted in literature for discriminating PD speech were not always found to be statistically significant. The automatic classification tasks to identify the most problematic speakers resulted in high precision and sensitivity by using two formant metrics simultaneously and in even higher performance by using acoustic dynamic parameters.

A method for the extraction of phonetically-rich triphone sentences

A method is proposed for compiling a corpus of phonetically-rich triphone sentences; i.e., sentences with a high variety of triphones, distributed in a uniform fashion. Such a corpus is of interest for a wide range of contexts, from automatic speech recognition to speech therapy. We evaluated this method by building phonetically-rich corpora for Brazilian Portuguese. The data employed comes from Wikipedias dumps, which were converted into plain text, segmented and phonetically transcribed. The method consists of comparing the distance between the triphone distribution of the available sentences to an ideal uniform distribution, with equiprobable triphones. A greedy algorithm was implemented to recognize and evaluate the distance among sentences. A heuristic metric is proposed for pre-selecting sentences for the algorithm, in order to quicken its execution. The results show that, by applying the proposed metric, one can build corpora with more uniform triphone distributions.

Automatic Annotation of Disfluent Speech in Children’s Reading Tasks

The automatic evaluation of reading performance of children is an important alternative to any manual or 1-on-1 evaluation by teachers or tutors. To do this, it is necessary to detect several types of reading miscues. This work presents an approach to annotate reading speech while detecting false-starts, repetitions and mispronunciations, three of the most common disfluencies. Using speech data of 6–10 year old children reading sentences and pseudowords, we apply a two-step process: first, an automatic alignment is performed to get the best possible word-level segmentation and detect syllable based false-starts and word repetitions by using a strict FST (Finite State Transducer); then, words are classified as being mispronounced or not through a likelihood measure of pronunciation by using phone posterior probabilities estimated by a neural network. This work advances towards getting the amount and severity of disfluencies to provide a reading ability score computed from several sentence reading tasks.

From European Portuguese to Portuguese Sign Language

Several efforts have been done towards the development of platforms that allow the translation of specific sign languages to the correspondent spoken language (and vice-versa). In this (demo) paper, we describe a freely available system that translates, in real time, European Portuguese (text) into Portuguese Sign Language (LGP), by using an avatar. We focus in how some linguistic elements are dealt in LGP, and in the Natural Language Processing (NLP) step implemented in our system. The system’s interface will be used to demonstrate it. Although only a small set of linguistic phenomena are implemented, it can be seen how the system copes with it.

Automatically Recognising European Portuguese Children’s Speech

This paper reports findings from an analysis of errors made by an automatic speech recogniser trained and tested with 3-10-year-old European Portuguese childrens speech. We expected and were able to identify frequent pronunciation error patterns in the childrens speech. Furthermore, we were able to correlate some of these pronunciation error patterns and automatic speech recognition errors. The findings reported in this paper are of phonetic interest but will also be useful for improving the performance of automatic speech recognisers aimed at children representing the target population of the study.

Rule-Based Algorithms for Automatic Pronunciation of Portuguese Verbal Inflections

The correct automatic pronunciation of words is a nontrivial problem, even for inflexions of Portuguese verbs, and has not been systematically solved yet, if verbal irregularity is taken into account. The purpose of this work is to enhance a grapheme-to-phoneme system with a verb pronunciation system for both varieties of Portuguese, Brazilian (BP) and European (EP), given only its infinitive form. The most common verbs for BP and EP (1000 and 2600 respectively) constituted our database to test the pronunciation system. A detailed and systematic analysis of regular and non-regular pronunciation forms of the inflected verbs was performed, and an index of irregularity for verb pronunciation is proposed. A rule-based algorithm to pronounce all inflexions according to verb paradigms is also described. The defined paradigms are, with a high level of certainty, representative of all the verbs for Portuguese.

Syllable structure in dysfunctional Portuguese children's speech

The goal of this work is to investigate whether children with speech dysfunctions (SD) show a deficit in planning some Portuguese syllable structures (PSS) in continuous speech production. Knowledge of which aspects of speech production are affected by SD is necessary for efficient improvement in the therapy techniques. The case-study is focused on PSS as C1C2V syllable sequences (consonant-consonant-vowel), in which C2 is [l] or [r]. To identify specific speech patterns that are sensitive to SD, coarticulation effects using formant trajectories, intensity, and durational structure are investigated. To explore the characteristics of continuous speech processes in SD speech output, the methodology uses acoustic analysis. Preliminary findings show systematic specific coarticulation in the child with SD when compared to the normal speech (NS) child. This also suggests that the traditional focus on a single word production in the SD assessment needs to be modified to allow more detailed consideration of speech production in continuous speech. It is the purpose of the authors in the future to develop an application that can be an optimal start for SD treatment/counselling programmes. The work reported here proves the importance of clinic linguistic knowledge in that way. This study is the result of a multidisciplinary-team whose work allies linguist, clinical therapy and engineering knowledge.