Nathan D. Nichols

Believable performance agents for interactive conversations

As computers become more integrated into our everyday lives, they will need to be able to interact with us within the context of our world as well as theirs. While it is unlikely that we will ever want all interactions with a computer to mimic dialogs with other people, it is clear that they will need to be able to engage in coherent, compelling conversations with people who are not thinking of them as machines. Our effort, and the system described below, is aimed at approaching this goal by creating a framework for believable performance agents within the content of interactive theatrical experiences.

Information Finding with Robust Entity Detection: The Case of an Online News Reader

Journalists and editors work under tight deadlines and are forced to gather as much background and details as they can about a particular situation or event. They have to keep track of useful sources and they have to be able to record what aspects and what portions of the source provided useful information.

Pivot: Automatically Offering Information and Services to Real-World Shoppers

Shoppers with an Internet-enabled computer have a wealth of product information available to them. By browsing to a variety of Websites, users can conduct searches and compare prices, read reviews, and learn more about a product. These sites are pivot points for a user; once they are at Amazon.coms landing page, for example, they can navigate outwards to a million different products. The idiom of browsing to a central page for a site and then navigating outwards is acceptable when browsing is convenient, with large displays and useful input devices. This process becomes inconvenient, however, when the user is out and about in the world. We have built a system, Pivot, that uses physical objects as pivot points for the user. Specifically, Pivot uses the 1-D barcodes present on every product to deliver powerful services and options to a user on his or her cellphone. These services are chosen to be most useful to a user in the moment and trying to make a purchase decision. This paper describes the motivations for the system, the system itself, its current real-world deployment, and our intended future work.

Learning to gesture: applying appropriate animations to spoken text

We propose a machine learning system that learns to choose human gestures to accompany novel text. The system is trained on scripts comprised of speech and animations that were hand-coded by professional animators and shipped in video games. We treat this as a text-classification problem, classifying speech as corresponding with specific classes of gestures. We have built and tested two separate classifiers. The first is trained simply on the frequencies of different animations in the corpus. The second extracts text features from each script, and maps these features to the gestures that accompany the script. We have experimented with using a number of features of the text, including n-grams, emotional valence of the text, and parts-of-speech. Using a naïve Bayes classifier, the system learns to associate these features with appropriate classes of gestures. Once trained, the system can be given novel text for which it will attempt to assign appropriate gestures. We examine the performance of the two classifiers by using n-fold cross-validation over our training data, as well as two user studies of subjective evaluation of the results. Although there are many possible applications of automated gesture assignment, we hope to apply this technique to a system that produces an automated news show.