Bassam Jabaian

Combination of stochastic understanding and machine translation systems for language portability of dialogue systems

In this paper, several approaches for language portability of dialogue systems are investigated with a focus on the spoken language understanding (SLU) component. We show that the use of statistical machine translation (SMT) can greatly reduce the time and cost of porting an existing system from a source to a target language. Using automatically translated training data we study phrase-based machine translation as an alternative to conditional random fields for conceptual decoding to compensate for the loss of a precise concept-word alignment. Also two ways to increase SLU robustness to translation errors (smeared training data and translation post-editing) are shown to improve performance when test data are translated then decoded in the source language. Overall the combination of all these approaches allows to reduce even further the concept error rate. Experiments were carried out on the French MEDIA dialogue corpus with a subset manually translated into Italian.

Comparison and Combination of Lightly Supervised Approaches for Language Portability of a Spoken Language Understanding System

Portability of a spoken dialogue system (SDS) to a new domain or a new language is a hot topic as it may imply gains in time and cost for building new SDSs. In particular in this paper we investigate several fast and efficient approaches for language portability of the spoken language understanding (SLU) module of a dialogue system. We show that the use of statistical machine translation (SMT) can reduce the time and the cost of porting a system from a source to a target language. For conceptual decoding, a state-of-the-art module based on conditional random fields (CRF) is used and a new approach based on phrase-based statistical machine translation (PB-SMT) is also evaluated. The experimental results show the efficiency of the proposed methods for a fast and low cost SLU language portability. In addition, we propose two methods to increase SLU robustness to translation errors. Overall, it is shown that the combination of all these approaches can further reduce the concept error rate. While most of the experiments in this paper deal with portability from French to Italian (given the availability of the Media French corpus and its subset manually translated into Italian), a validation of our methodology is eventually proposed in Arabic.

Online adaptative zero-shot learning spoken language understanding using word-embedding

Many recent competitive state-of-the-art solutions for understanding of speech data have in common to be probabilistic and to rely on machine learning algorithms to train their models from large amount of data. The difficulty remains in the cost and time of collecting and annotating such data, but also to update the existing models to new conditions, tasks and/or languages. In the present work an approach based on a zeroshot learning method using word embeddings for spoken language understanding is investigated. This approach requires no dedicated data. Large amounts of un-annotated and unstructured found data are used to learn a continuous space vector representation of words, based on neural network architectures. Only the ontological description of the target domain and the generic word embedding features are then required to derive the model used for decoding. In this paper, we extend this baseline with an online adaptative strategy allowing to refine progressively the initial model with only a light and adjustable supervision. We show that this proposition can significantly improve the performance of the spoken language understanding module on the second Dialog State Tracking Challenge (DSTC2) datasets.

Adversarial Bandit for online interactive active learning of zero-shot spoken language understanding

Many state-of-the-art solutions for the understanding of speech data have in common to be probabilistic and to rely on machine learning algorithms to train their models from large amount of data. The difficulty remains in the cost of collecting and annotating such data. Another point is the time for updating an existing model to a new domain. Recent works showed that a zero-shot learning method allows to bootstrap a model with good initial performance. To do so, this method relies on exploiting both a small-sized ontological description of the target domain and a generic word-embedding semantic space for generalization. Then, this framework has been extended to exploit user feedbacks to refine the zero-shot semantic parser parameters and increase its performance online. In this paper, we propose to drive this online adaptive process with a policy learnt using the Adversarial Bandit algorithm Exp3. We show, on the second Dialog State Tracking Challenge (DSTC2) datasets, that this proposition can optimally balance the cost of gathering valuable user feedbacks and the overall performance of the spoken language understanding module.

A unified framework for translation and understanding allowing discriminative joint decoding for multilingual speech semantic interpretation

HighlightsA comparison between the methods used for speech translation and understanding.A unified framework for translation and understanding.A discriminative joint decoding for multilingual speech semantic interpretation.The proposition is competitive with state-of-the-art techniques.The framework can be generalized to other components of a dialogue system. Probabilistic approaches are now widespread in most natural language processing applications and selection of a particular approach usually depends on the task at hand. Targeting speech semantic interpretation in a multilingual context, this paper presents a comparison between the state-of-the-art methods used for machine translation and speech understanding. This comparison justifies our proposition of a unified framework for both tasks based on a discriminative approach. We demonstrate that this framework can be used to perform a joint translation-understanding decoding which allows to combine, in the same process, translation and semantic tagging scores of a sentence. A cascade of finite-state transducers is used to compose the translation and understanding hypothesis graphs (1-bests, word graphs or confusion networks). Not only this proposition is competitive with the state-of-the-art techniques but also its framework is even more attractive as it can be generalized to other components of human-machine vocal interfaces (e.g. speech recognizer) so as to allow a richer transmission of information between them.

Generalization of discriminative approaches for speech language understanding in a multilingual context

Probabilistic approaches are now widespread in the various applications of natural language processing and elicitation of a particular approach usually depends on the task at hand. Targeting multilingual interpretation of speech, this paper presents a comparison between the state-of-the-art methods used for machine translation and speech understanding. This comparison justifies our proposition of a unified framework to perform a joint decoding which translates a sentence and assigns semantic tags to this translation in the same process. The decoding is achieved using a cascade of finite-state transducers allowing to compose translation and understanding hypothesis graphs. This representation is favorable as it can be generalized to allow rich transmission of information between the components of a human-machine vocal interface.

Integration and evaluation of social competences such as humor in an artificial interactive agent

In this paper, we present a brief overview of our ongoing work about artificial interactive agents and their adaptation to users. Several possibilities to introduce humorous productions in a spoken dialog system are investigated in order to enhance naturalness during social interactions between the agent and the user. We finally describe our plan on how neuroscience will help to better evaluate the proposed systems, both objectively and subjectively.