Stefan Marti

Physical embodiments for mobile communication agents

This paper describes a physically embodied and animated user interface to an interactive call handling agent, consisting of a small wireless animatronic device in the form of a squirrel, bunny, or parrot. A software tool creates movement primitives, composes these primitives into complex behaviors, and triggers these behaviors dynamically at state changes in the conversational agents finite state machine. Gaze and gestural cues from the animatronics alert both the user and co-located third parties of incoming phone calls, and data suggests that such alerting is less intrusive than conventional telephones.

I/O brush: beyond static collages

I/O Brush is our ongoing effort to empower people to create new expressions and meanings by painting with attributes of everyday objects and movements in their physical world. Using examples from our case studies with kindergarteners and artists, we discuss I/O Brushs most distinguishing features, its dynamic ink and history functions, and how they enable people to invent new expressions and meaning making with objects in their physical environment.

Ghost fingers: a hybrid approach to the interaction with remote displays

In this paper, we describe a novel interaction method called Ghost Fingers, which enables efficient and intuitive switching between keyboard and multi-touch input on systems where the display is out of arms reach. In addition, Ghost Fingers provides a translucent real-time visualization of the fingers and hands on the remote display, creating a closed interaction loop that enables direct manipulation even on remote displays. Our solution includes a wireless keyboard with attached imaging sensor that is used to both determine the position of the users hand and fingers, and to provide a real-time translucent overlay of hand and fingers over the remote UI.

DRIVE: Directly Reaching Into Virtual Environment with bare hand manipulation behind mobile display

We present DRIVE, an interaction method that allows a user to manipulate virtual content by reaching behind a mobile display device such as a cellphone, tablet PC, etc. Unlike prior work that uses front volume as well as front, side, and back surfaces, DRIVE utilizes the back volume of the device. Together with face tracking, out system creates the illusion that the users hand is co-located with virtual volumetric content.