Nitin Sawhney

Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments

Mobile workers need seamless access to communication and information services while on the move. However, current solutions overwhelm users with intrusive interfaces and ambiguous notifications. This article discusses the interaction techniques developed for Nomadic Radio, a wearable computing platform for managing voice and text-based messages in a nomadic environment. Nomadic Radio employs an auditory user interface, which synchronizes speech recognition, speech synthesis, nonspeech audio, and spatial presentation of digital audio, for navigating among messages as well as asynchonous notific ation of nely arrived messages. Emphasis is placed on an auditory modality as Nomadic Radio is designed to be used while performing other tasks in a users everyday environment; a range of auditory cues provides peripheral awareness of incoming messages. Notification is adaptive and cntext sensitive; messages are presented as more or less obtrsive based on importance inferred from content filtering, whether the user is engaged in conversation and his or her own recent responses to prior messages. Auditory notifications are dynamically scaled from ambient sound through recorded voice cues up to message summaries. Iterative design and a preliminary user evaluation suggest that audio is an appropriate medium for mobile messaging, but that care must be taken to minimally intrude on the wearers social and physical environment.

Teaching and learning as multimedia authoring: the classroom 2000 project

We view college classroom teaching and learning as a multimedia authoring activity. The classroom provides a rich setting in which a number of different forms of communication co-exist, such as speech, writing and projected images. Much of the information in a lecture is poorly recorded or lost currently. Our hypothesis is that tools to aid in the capture and subsequent access of classroom information will enhance both the learning and teaching experience. To test that hypothesis, we initiated the Classroom 2000 project at Georgia Tech. The purpose of the project is to apply ubiquitous computing technology to facilitate automatic capture, integration and access of multimedia information in the educational setting of the university classroom. In this paper, we discuss various prototype tools we have created and used in a variety of courses and provide an initial evaluation of the acceptance and effectiveness of the technology. We also share some lessons learned in applying ubiquitous computing technology in a real setting.

HyperCafe: narrative and aesthetic properties of hypervideo

HyperCafe is an experimental hypermedia prototype, developed as an illustration of a general hypervideo system. This program places the user in a virtual cafe, composed primarily of digital video clips of actors involved in fictional conversations in the cafe; HyperCafe allows the user to follow different conversations, and offers dynamic opportunities of interaction via temporal, spatio-temporal and textual links to present alternative narratives. Textual elements are also present in the form of explanatory text, contradictory subtitles, and intruding narratives. Based on our work with HyperCafe, we discuss the components and a framework for hypervideo structures, along with the underlying aesthetic considerations.

Dynomite: a dynamically organized ink and audio notebook

Dynomite is a portable electronic notebook for the capture and retrieval of handwritten and audio notes. Dynomite provides a casual, paper like interface, allowing the user to write or draw on a screen resembling an ordinary paper notebook. Indexing is provided by allowing users to attach properties, or data types, to specific ink, and by allowing the user to associate text keywords with pages of notes. Dynomite notes can be organized into sub notebooks based on these indexes. In addition, Dynomite records audio synchronously with notes and allows users to mark or highlight significant sections, so that they may be easily located and replayed. In order to understand how well the features of Dynomite support natural note taking practices, we conducted usage studies, which showed that Dynomites indexing capabilities clearly improve the ability to find information in notes. (4 pages)

Nomadic radio: scaleable and contextual notification for wearable audio messaging

Mobile workers need seamless access to communication andinformation services on portable devices. However current solutionsoverwhelm users with intrusive and ambiguous notifications. In thispaper, we describe scaleable auditory techniques and a contextualnotification model for providing timely information, whileminimizing interruptions. Users actions influence local adaptationin the model. These techniques are demonstrated in Nomadic Radio,an audio-only wearable computing platform.

Authoring and navigating video in space and time

A formal methodology is needed to integrate and exchange spatial and temporal properties in hypermedia and hypertext. We propose a generic framework to structure and dynamically present a new form of video- and text-based media called hypervideo. We developed a Hypervideo Engine and produced an experimental hypermedia work, HyperCafe, to illustrate the general properties and aesthetic techniques possible in such a medium.

Voices beyond walls: the role of digital storytelling for empowering marginalized youth in refugee camps

In this paper we consider the role of digital storytelling for creative empowerment of marginalized youth, through a threeyear program of workshops conducted in the context of Palestinian refugee camps in the West Bank and East Jerusalem. We highlight the current situation regarding arts education and cultural production, and the challenging environment experienced by the youth there. We then describe the Voices Beyond Walls program and key lessons learned to suggest novel tools and practices that may support broader adoption of digital storytelling programs for marginalized youth in diverse global settings.

Aware Community Portals: Shared Information Appliances for Transitional Spaces

Abstract: People wish to maintain a level of awareness of timely information, including presence of others in the workplace and other social settings. We believe this provides better exchange, coordination and contact within a community, especially as people work in asynchronous times and distributed locations. The challenge is to develop lightweight techniques for awareness, interaction and communication using shared information appliances. In this paper, we describe the design of an exploratory responsive display projected within a shared workspace at the MIT Media Lab. The system uses visual sensing to provide relevant information and constructs traces of people’s activity over time. Such aware portals may be deployed in casual workplace domains, distributed workgroups, and everyday public spaces.

Nomadic radio: a spatialized audio environment for wearable computing

This paper describes an on-going research project at the MIT Media Lab, exploring the use of auditory I/O as a primary interaction modality for wearable computing. Nomadic Radio is a framework developed for use on a wearable audio platform. It presents timely information and permits communication within a spatialized audio environment. The contextual state of the user indicated by time of day, physical positioning, scheduled tasks, and message content, is used to model information presented in the users listening space. We are evaluating techniques for the design of spatial audio streams, speech I/O, and auditory cues in the interface. Issues related to asynchronous communication and peripheral awareness are also being considered.

ESPACE 2: an experimental hyperaudio environment

Espace 2 is a prototype system for navigation of hyper-linked audio information in an immersive audio-only environment. In this paper, we propose several essential design concepts for audio-only computing environments. We will describe a hyperaudio system based on the prior design principles and discuss an evaluation of the preliminwy prototype.