Ivan Meza

The DIME Project

In this paper a general description and current state of the project Dialogos Multimodales Inteligentes en Espanol (DIME) -Intelligent Multimodal Dialogs in Spanish- is presented. The purpose of the project is to develop a multimodal conversational agent with spoken input and output facilities in Spanish in a design oriented domain: kitchen design. In this paper, the state of the project, current results, an overview of the prototype system and future work are presented.

Dialogue model specification and interpretation for intelligent multimodal HCI

A methodology and programming environment for the specification and interpretation of dialogue models for grounded multimodal interaction is presented. This conceptual framework permits the declarative specification of complex interactive systems with multimodal input and output, including speech, computer vision and motor behavior. We first introduce the present notion of dialogue model with its motivation on the structure of conversation. Then, the specification and interpretation of dialogue models is presented and discussed. We also present a cognitive architecture for the construction of intelligent Human-Computer Interaction (HCI) applications within this conceptual framework. The paper concludes with references to working systems, demos and work in progress built within the present framework.

Development of a tour-guide robot using dialogue models and a cognitive architecture

In this paper, we present the development of a tour-guide robot that conducts a poster session through spoken Spanish. The robot is able to navigate around its environment, visually identify informational posters, and explain sections of the posters that users request via pointing gestures. We specify the task by means of dialogue models. A dialogue model defines conversational situations, expectations and robot actions. Dialogue models are integrated into a novel cognitive architecture that allow us to coordinate both human-robot interaction and robot capabilities in a flexible and simple manner. Our robot also incorporates a confidence score on visual outcomes, the history of the conversation and error prevention strategies. Our initial evaluation of the dialogue structure shows the reliability of the overall approach, and the suitability of our dialogue model and architecture to represent complex human-robot interactions, with promising results.

SitLog: A Programming Language for Service Robot Tasks

In this paper we present SitLog: a declarative situation-oriented logical language for programming situated service robot tasks. The formalism is task and domain independent, and can be used in a wide variety of settings. SitLog can also be seen as a behaviour engineering specification and interpretation formalism to support action selection by autonomous agents during the execution of complex tasks. The language combines the recursive transition network formalism, extended with functions to express dynamic and contextualized task structures, with a functional language to express control and content information. The SitLog interpreter is written in Prolog and SitLogs programs follow closely the Prolog notation, permitting the declarative specification and direct interpretation of complex applications in a modular and compact form. We discuss the structure and representation of service robot tasks in practical settings and how these can be expressed in SitLog. The present framework has been tested in the service robot Golem-II+ using the specification and programming of the typical tasks which require completion in the RoboCup@Home Competition.

Concept and Functional Structure of a Service Robot

In this paper, we present a concept of service robot and a framework for its functional specification and implementation. The present discussion is grounded in Newells system levels hierarchy which suggests organizing robotics research in three different layers, corresponding to Marrs computational, algorithmic and implementation levels, as follows: (1) the service robot proper, which is the subject of the present paper, (2) perception and action algorithms, and (3) the systems programming level. The concept of a service robot is articulated in practice through the introduction of a conceptual model for particular service robots; this consists of the specification of a set of basic robotic behaviours and a number of mechanisms for assembling such behaviours during the execution of complex tasks. The model involves an explicit representation of the task structure, allowing for deliberative reasoning and task management. The model also permits distinguishing between a robots competence and performance, along the lines of Chomskys corresponding distinction. We illustrate how this model can be realized in practice with two composition modes that we call static and dynamic; these are illustrated with the Restaurant Test and the General Purpose Service Robot Test of the RoboCup@Home competition, respectively. The present framework and methodology has been implemented in the robot Golem-II+, which is also described. The paper is concluded with an overall reflection upon the present concept of a service robot and its associated functional specifications, and the potential impact of such a conceptual model in the study, development and application of service robots in general.

Integration of the Multi-DOA Estimation Functionality to Human-Robot Interaction

Sound source localization is important in human interaction, such as in locating the origin of long-distance calls or facing other humans while in a conversation. It is of interest to apply such functionality to the core of human-robot interaction (HRI) and investigate its benefits, if any. In this paper, we propose three strategies for how to integrate the functionality of multiple directions-of-arrival (multi-DOA) estimation with a common scenario, in which the robot acts as a waiter while applying audio source localization. The proposed strategies are: a) the robot locates calls from users at a relatively long distance; b) the robot faces the user when taking the order; and c) the robot announces whether the acoustic environment is not conducive to understanding a speech command (mainly where more than one user speaks at once). It was seen that users react favourably to the functionality, and that it even has a noticeable influence on the success of the interaction.

Localization of sound sources in robotics: A review

Abstract Sound source localization (SSL) in a robotic platform has been essential in the overall scheme of robot audition. It allows a robot to locate a sound source by sound alone. It has an important impact on other robot audition modules, such as source separation, and it enriches human–robot interaction by complementing the robot’s perceptual capabilities. The main objective of this review is to thoroughly map the current state of the SSL field for the reader and provide a starting point to SSL in robotics. To this effect, we present: the evolution and historical context of SSL in robotics; an extensive review and classification of SSL techniques and popular tracking methodologies; different facets of SSL as well as its state-of-the-art; evaluation methodologies used for SSL; and a set of challenges and research motivations.

On Indexicality, Direction of Arrival of Sound Sources, and Human-Robot Interaction

We present the use of direction of arrival (DOA) of sound sources as an index during the interaction between humans and service robots. These indices follow the notion defined by the theory of interpretation of signs by Peirce. This notion establishes a strong physical relation between signs (DOAs) and objects being signified in specific contexts. With this in mind, we have modeled the call at a distance to a robot as indexical in nature. These indices can be later interpreted as the position of the user and the user herself/himself. The relation between the call and the emitter is formalized in our framework of development of service robots based on the SitLog programming language. In particular, we create a set of behaviours based on direction of arrival information to be used in the programming of tasks for service robots. Based on these behaviours, we have implemented four tasks which heavily rely on them: following a person, taking attendance of a class, playing Marco-Polo, and acting as a waiter in a restaurant.

Contextual semantic processing for a spanish dialogue system using markov logic

Semantic processing is vital in a dialogue system for the language understanding stage. Recent approaches of semantic processing rely on machine learning methods to perform the task. These are more robust to errors from the speech recogniser. Although these approaches are built on the domain of the dialogue system they do not incorporate contextual information available in the dialogue system. In this paper, we explore the use of contextual information in the form of expectations of a dialogue system to perform semantic processing in a Spoken Dialogue System. We show the benefits on doing so, and propose a Markov Logic model which incorporates such information.

The Spanish Auxiliary Verb System in HPSG

This paper presents an analysis of the Spanish auxiliary verbs from a syntactic point of view instead a semantic analysis that is proposed by the traditional Spanish grammars. The syntactic context of the Spanish auxiliary verbs is clarified with the definition of five properties, which allow us to determinate if a verb behave as an auxiliary verb or not. The subject raising verb type defined in the formalism HPSG is used in order to capture the behavior of Spanish auxiliary verbs. We conclude with the analysis of a typical auxiliary verb, poder, as a case of study.