Lou Boves

Quantitative assessment of second language learners' fluency by means of automatic speech recognition technology

To determine whether expert fluency ratings of read speech can be predicted on the basis of automatically calculated temporal measures of speech quality, an experiment was conducted with read speech of 20 native and 60 non-native speakers of Dutch. The speech material was scored for fluency by nine experts and was then analyzed by means of an automatic speech recognizer in terms of quantitative measures such as speech rate, articulation rate, number and length of pauses, number of dysfluencies, mean length of runs, and phonation/time ratio. The results show that expert ratings of fluency in read speech are reliable (Cronbachs alpha varies between 0.90 and 0.96) and that these ratings can be predicted on the basis of quantitative measures: for six automatic measures the magnitude of the correlations with the fluency scores varies between 0.81 and 0.93. Rate of speech appears to be the best predictor: correlations vary between 0.90 and 0.93. Two other important determinants of reading fluency are the rate at which speakers articulate the sounds and the number of pauses they make. Apparently, rate of speech is such a good predictor of perceived fluency because it incorporates these two aspects.

Quantitative assessment of second language learners’ fluency: Comparisons between read and spontaneous speech

This paper describes two experiments aimed at exploring the relationship between objective properties of speech and perceived fluency in read and spontaneous speech. The aim is to determine whether such quantitative measures can be used to develop objective fluency tests. Fragments of read speech (Experiment 1) of 60 non-native speakers of Dutch and of spontaneous speech (Experiment 2) of another group of 57 non-native speakers of Dutch were scored for fluency by human raters and were analyzed by means of a continuous speech recognizer to calculate a number of objective measures of speech quality known to be related to perceived fluency. The results show that the objective measures investigated in this study can be employed to predict fluency ratings, but the predictive power of such measures is stronger for read speech than for spontaneous speech. Moreover, the adequacy of the variables to be employed appears to be dependent on the specific type of speech material investigated and the specific task performed by the speaker.

The pedagogy-technology interface in Computer Assisted Pronunciation Training

In this paper, we examine the relationship between pedagogy and technology in Computer Assisted Pronunciation Training (CAPT) courseware. First, we will analyse available literature on second language pronunciation teaching and learning in order to derive some general guidelines for effective training. Second, we will present an appraisal of various CAPT systems with a view to establishing whether they meet pedagogical requirements. In this respect, we will show that many commercial systems tend to prefer technological novelties to the detriment of pedagogical criteria that could benefit the learner more. While examining the limitations of todays technology, we will consider possible ways to deal with these shortcomings. Finally, we will combine the information thus gathered to suggest some recommendations for future CAPT.

A Spoken Dialog System for the Dutch Public Transport Information Service

In the Netherlands there is a nationwide premium rate telephone number that can be dialed to obtain information about various forms of public transport. In 1996 this number was called more than twelve million times. Human operators managed to handle only about nine million of these calls. In order to answer more of these calls, a spoken dialog system was developed to automate part of this service. The automation component concerns information about journeys between two train stations.The starting point of our research was an existing German information system. This system was ported to Dutch. A bootstrapping method was used to collect the data, which in turn were used to improve the system itself.

Different aspects of expert pronunciation quality ratings and their relation to scores produced by speech recognition algorithms

The ultimate aim of the research reported on here is to develop an automatic testing system for Dutch pronunciation. In the experiment described in this paper automatic scores of telephone speech produced by native and non-native speakers of Dutch are compared with specific, i.e., temporal and segmental, and global pronunciation ratings assigned by three groups of experts: three phoneticians and two groups of three speech therapists. The goals of this experiment are to determinutee (1) whether specific expert ratings of pronunciation quality contribute to our understanding of the relation between human pronunciation scores and machine scores of speech quality, (2) whether different expert groups assign essentially different ratings, and (3) to what extent rater pronunciation scores can be predicted on the basis of automatic scores. The results show that collecting specific ratings along with overall ones leads to a better understanding of the relation between human and automatic pronunciation assessment. Furthermore, after normalization no considerable differences are observed between the ratings by the three expert groups. Finally, it appears that the speech quality scores produced by our speech recognizer can predict expert pronunciation ratings with a high degree of accuracy.

On temporal aspects of turn taking in conversational dialogues

Abstract In this short communication we show how shallow annotations in large speech corpora can be used to derive data about the temporal aspects of turn taking. Within the limitations of such a speech corpus, we show that the average durations of between-turn pauses made by speakers in a dyad are statistically related, and our data suggest the existence of gender effects in the temporal aspects of turn taking. Also, clear differences in turn taking behaviour between face-to-face and telephone dialogues can be detected using shallow analyses. We discuss the most important limitations imposed by the shallowness of the annotations in large corpora, and the possibility for enriching those annotations in a semi-automatic iterative manner.

Evaluating discourse-based answer extraction for why -question answering

30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007)

Acoustic reduction in conversational Dutch: A quantitative analysis based on automatically generated segmental transcriptions

Abstract In spontaneous, conversational speech, words are often reduced compared to their citation forms, such that a word like yesterday may sound like [ ’ j e ʃ e I ] . The present chapter investigates such acoustic reduction. The study of reduction needs large corpora that are transcribed phonetically. The first part of this chapter describes an automatic transcription procedure used to obtain such a large phonetically transcribed corpus of Dutch spontaneous dialogues, which is subsequently used for the investigation of acoustic reduction. First, the orthographic transcriptions were adapted for automatic processing. Next, the phonetic transcription of the corpus was created by means of a forced alignment using a lexicon with multiple pronunciation variants per word. These variants were generated by applying phonological and reduction rules to the canonical phonetic transcriptions of the words. The second part of this chapter reports the results of a quantitative analysis of reduction in the corpus on the basis of the generated transcriptions and gives an inventory of segmental reductions in standard Dutch. Overall, we found that reduction is more pervasive in spontaneous Dutch than previously documented.

An overview of the CAVE project research activities in speaker verification

This article presents an overview of the research activities carried out in the European CAVE project, which focused on text-dependent speaker verification on the telephone network using whole word Hidden Markov Models. It documents in detail various aspects of the technology and the methodology used within the project. In particular, it addresses the issue of model estimation in the context of limited enrollment data and the problem of a posteriori decision threshold setting. Experiments are carried out on the realistic telephone speech database SESP. State-of-the-art performance levels are obtained, which validates the technical approaches developed and assessed during the project as well as the working infrastructure which facilitated cooperation between the partners.

Effective error recovery strategies for multimodal form-filling applications

The goal of the research described in this article is to determine in what way speech recognition errors can be handled best in a multimodal form-filling interface. Besides two well-known error correction mechanisms (re-speaking the value and choosing the correct value from a list of alternatives), the interface offers a novel correction mechanism in which the user selects the first letter of the target word from a soft-keyboard, after which the utterance is recognized once again, with a limited language model and lexicon. The multimodal interface that was used is a web-based form-filling GUI, extended with a speech overlay, which allows for pen and speech input. The effectiveness and efficiency of the error correction mechanisms, the error correction strategies that are applied by the users and the effects on user satisfaction were studied in an evaluation in which the interface was tested in two conditions: in one condition (LIST), the interface provides only re-speaking and the alternatives list as error correction facilities. In the other condition (LETTER), the interface provides the soft-keyboard technique as an additional error correction facility. The study shows that error correction was more effective in the LETTER condition than in the LIST condition. The Keyboard correction facility enables the users to solve errors that could not be solved using the Re-speak method or by choosing from a list of alternatives. In spite of its low effectiveness, subjects initially attempted to use Re-speaking for error correction in both interfaces. However, we also found that subjects rapidly learned to choose the most effective option (Keyboard) immediately as they gain experience. The user satisfaction turned out to be higher for the LETTER interface than for the LIST interface: subjects considered the LETTER interface to be more useful and less frustrating and they felt more in control. As a result, most subjects clearly preferred the LETTER interface.