Joyce Yue Chai

Natural language dialogue for personalized interaction

T he pragmatic goal of natural language (NL) and multimodal interfaces (speech recognition, keyboard entry, pointing, among others) is to enable ease-of-use for users/customers in performing more sophisticated human-computer interactions (HCI). NL research attempts to define extensive discourse models that in turn provide improved models of context-enabling HCI and personalization. Customers have the initiative to Technologies that successfully recognize and react to spoken or typed words are key to true personalization. Front-and back-end systems must respond in accord, and one solution may be found somewhere in the middle(ware).

Context-based multimodal input understanding in conversational systems

In a multimodal human-machine conversation, user inputs are often abbreviated or imprecise. Sometimes, merely fusing multimodal inputs together cannot derive a complete understanding. To address these inadequacies, we are building a semantics-based multimodal interpretation framework called MIND (Multimodal Interpretation for Natural Dialog). The unique feature of MIND is the use of a variety of contexts (e.g., domain context and conversation context) to enhance multimodal fusion. In this paper we present a semantically rich modeling scheme and a context-based approach that enable MIND to gain a full understanding of user inputs, including ambiguous and incomplete ones.

The Role of a Natural Language Conversational Interface in Online Sales: A Case Study

This paper describes the evaluation of a natural language dialog-based navigation system (HappyAssistant) that helps users access e-commerce sites to find relevant information about products and services. The prototype system leverages technologies in natural language processing and human-computer interaction to create a faster and more intuitive way of interacting with websites, especially for less experienced users. The result of a comparative study shows that users prefer the natural language-enabled navigation two to one over the menu driven navigation. In addition, the study confirmed the efficiency of using natural language dialog in terms of the number of clicks and the amount of time required to obtain the relevant information. In the case study, as compared to the menu driven system, the average number of clicks used in the natural language system was reduced by 63.2% and the average time was reduced by 33.3%.

A conversational interface for online shopping

We present a deployed, conversational dialog system that assists users in finding computers based on their usage patterns and constraints on specifications. We discuss findings from a market survey and two user studies. We compared our system to a directed dialog system and a menu driven navigation system. We found that the conversational interface reduced the average number of clicks by 63% and the average interaction time by 33% over a menu driven search system. The focus of our continuing work includes developing a dynamic, adaptive dialog management strategy, robustly handling user input and improving the user interface.

Dynamic User Level and Utility Measurement for Adaptive Dialog in a Help-Desk System

The learning and self-adaptive capability in dialog systems has become increasingly important with the advances in a wide range of applications. For any application, particularly the one dealing with a technical domain, the system should pay attention to not only the user experience level and dialog goals, but more importantly, the mechanism to adapt the system behavior to the evolving state of the user. This paper describes a methodology that first identifies the user experience level and utility metrics of the goal and sub-goals, then automatically adjusts those parameters based on discourse history and thus directs adaptive dialog management.

Semantics-based representation for multimodal interpretation in conversational systems

To support context-based multimodal interpretation in conversational systems, we have developed a semantics-based representation to capture salient information from user inputs and the overall conversation. In particular, we present three unique characteristics: fine-grained semantic models, flexible composition of feature structures, and consistent representation at multiple levels. This representation allows our system to use rich contexts to resolve ambiguities, infer unspecified information, and improve multimodal alignment. As a result, our system is able to enhance understanding of multimodal inputs including those abbreviated, imprecise, or complex ones.

Conversational sales assistant for online shopping

Websites of businesses should accommodate both customer needs and business requirements. Traditional menu-driven navigation and key word search do not allow users to describe their intentions precisely. We have developed a conversational interface to online shopping that provides convenient, personalized access to information using natural language dialog. User studies show significantly reduced length of interactions in terms of time and number of clicks in finding products. The core dialog engine is easily adaptable to other domains.