Nuno J. Mamede

Recovering capitalization and punctuation marks for automatic speech recognition: Case study for Portuguese broadcast news

The following material presents a study about recovering punctuation marks, and capitalization information from European Portuguese broadcast news speech transcriptions. Different approaches were tested for capitalization, both generative and discriminative, using: finite state transducers automatically built from language models; and maximum entropy models. Several resources were used, including lexica, written newspaper corpora and speech transcriptions. Finite state transducers produced the best results for written newspaper corpora, but the maximum entropy approach also proved to be a good choice, suitable for the capitalization of speech transcriptions, and allowing straightforward on-the-fly capitalization. Evaluation results are presented both for written newspaper corpora and for broadcast news speech transcriptions. The frequency of each punctuation mark in BN speech transcriptions was analyzed for three different languages: English, Spanish and Portuguese. The punctuation task was performed using a maximum entropy modeling approach, which combines different types of information both lexical and acoustic. The contribution of each feature was analyzed individually and separated results for each focus condition are given, making it possible to analyze the performance differences between planned and spontaneous speech. All results were evaluated on speech transcriptions of a Portuguese broadcast news corpus. The benefits of enriching speech recognition with punctuation and capitalization are shown in an example, illustrating the effects of described experiments into spoken texts.

Reengineering a Domain-Independent Framework for Spoken Dialogue Systems

Our work in this area started as a research project but when L2F joined TecnoVoz, a Portuguese national consortium including Academia and Industry partners, our focus shifted to real-time professional solutions. The integration of our domain-independent Spoken Dialogue System (SDS) framework into commercial products led to a major reengineering process. n nThis paper describes the changes that the framework went through and that deeply affected its entire architecture. The communication core was enhanced, the modules interfaces were redefined for an easier integration, the SDS deployment process was optimized and the framework robustness was improved. The work was done according to software engineering guidelines and making use of design patterns.

Starting to Cook a Coaching Dialogue System in the Olympus framework

This paper presents the COOKCOACH, a system that guides the user while cooking a recipe during a spoken dialogue coaching session. We describe our experience creating the first version of our coaching dialogue system that helps the user cook a selected dish.We use the CMU dialogue systems framework OLYMPUS. The main challenge we faced was the change of paradigm: instead of the system being driven by the user, the user is guided by the system. The result is a system capable of dictating generic tasks to the user. The methodology allows us to create different systems in diverse domains for similar guidance tasks that support the user while s/he performs some procedural task. The use of ontologies to facilitate the creation of new systems is briefly explored.

Using System Expectations to Manage User Interactions

This paper presents a new approach to parse multiple data types in Dialogue Systems. In its initial version, our spoken dialogue systems platform had a single and generic parser. However, when developing two new systems, the parsers complexity increased and data types, like numbers, dates and free text messages, were not correctly interpreted. The solution we present to cope with these problems allows the system to rely on expectations about the flow of the dialogue based on the dialogue history and context. Because these expectations guide the parsing process, a positive impact is achieved in the recognition of objects in the users utterance. However, if the user fails to match the systems expectations, for instance by changing the focus of the conversation, the system is still capable of understanding the input and recognizing the referred objects.

Expanding SNePS Capabilities with LORE

We briefly describe LORE, a logic with four values, the traditional truth values T and F, and two “Unknown” values, allowing to differentiate between knowing that nothing is known, and not knowing (with the available resources) whether it is known. A computer system based on LORE has the capability to remember all the paths followed during an attempt to answer a question. For each path, it records the used hypotheses (the hypotheses that constitute the path), the missing hypotheses (when the path did not lead to an answer), and why they were assumed missing. A number of examples of the use of LORE are discussed, and it is shown that SNePS capabilities can be expanded if LORE is accepted as the logic underlying its inferences.

SnePSR—A SNePS with resources

Abstract We present an evolution of SNePS, the SnePSR (from SNePS with resources) knowledge representation/reasoner system. SnePSR is an intelligent resource-bounded reasoner that allows several resource spending strategies. Since no a priori commitments are made about the way resources are spent, the process of consuming resources can be used to model non-omniscient, non-exhaustive reasoners. SnePSR combines the introduction of resources with the capability to produce conditional answers, which explicitly reveals the impediments that are responsible for the absence of a definite answer. SnePSR avoids two problems that most of the programs, trying to behave intelligently, suffer from: (1) never take into account the fact that reasoning resources are limited; (2) remain silent whenever a definite answer cannot be produced. After briefly presenting how the characteristics that distinguish SNePSR have been incorporated into SNePS we present some case studies of interactions with SNePSR demonstrating some of ...

Reasoning with the Unknown

We define a logical system with four values, the traditional truth values T and F and two “Unknown” values. An inference system based on this logic has the capability to remember all the paths followed during an attempt to answer a question. For each path it records the used hypotheses (the hypotheses that constitute the path), the missing hypothesis (when the path did not lead to an answer), and why it was assumed as missing. The inference system takes special care with missing hypotheses that are contradictory with any hypothesis that is being considered. An inference system with these capabilities can report the answers found and the reasons that prevented the inference of other potential answers. This capability can be used to plan reasoning, to perform default reasoning, and to reason about its own knowledge.

Advances in Speech and Language Technologies for Iberian Languages

This paper presents the results of the analysis of a set prosodic parameters considered relevant for the expression of emotion in Spanish on a corpus of read aloud chat messages, and explores the application of the obtained results to generate emotional synthetic speech using a novel parametric approach. The obtained results show that the analysed parameters seem to be relevant for the differentiation among the considered emotions, but that its use in parametric synthesis does not offer yet the desired quality level, although better in any case than using corpus-based techniques.

Timetabling Using Demand Profiles

Timetabling problems can be extremely time-consuming when they are solved without any kind of computer assistance. Computer assistance can vary from some intuitive graphical interface to an automated timetabler. Although a good graphical interface may be suitable for small problems, when we consider medium-size or large-size problems only an automated tool can be useful. In this paper we introduce a new paradigm for automated timetabling based on models and techniques developed for scheduling. Scheduling concepts such as activity and resource are translated to the timetabling domain and a general Bardadyms scheduling method, named micro-opportunistic approach, is applied in this novel domain. This approach constructs schedules incrementally and always focus its attention on the most critical decisions, to avoid backtracking. This framework is constraint-based and object-oriented. These two methods allows the easy representation of the timetabling problem and the handling of new timetabling constraints, either “hard” (should be satisfied) or “soft” (should preferably be satisfied).

RR — An Intelligent Resource-Bounded Reasoner

Most of the programs developed by AI researchers, suffer from two problems: (1) they do not take into account the fact of reasoning resources are limited; (2) and they remain silent (no answer is produced) whenever a definite answer cannot be produced.