Bradley J. Rhodes

Augmented reality through wearable computing

Wearable computing moves computation from the desktop to the user. We are forming a community of networked, wearable-computer users to explore, over a long period, the augmented realities that these systems can provide. By adapting its behavior to the users changing environment, a body-worn computer can assist the user more intelligently, consistently, and continuously than a desktop system. A text-based augmented reality, the Remembrance Agent, is presented to illustrate this approach. Video cameras are used both to warp the visual input (mediated reality) and to sense the users world for graphical overlay. With a camera, the computer could track the users finger to act as the systems mouse; perform face recognition; and detect passive objects to overlay 2.5D and 3D graphics onto the real world. Additional apparatus such as audio systems, infrared beacons for sensing location, and biosensors for learning about the wearers affect are described. With the use of input from these interface devices and sensors, a long-term goal of this project is to model the users actions, anticipate his or her needs, and perform a seamless interaction between the virtual and physical environments.

Just-in-time information retrieval agents

A just-in-time information retrieval agent (JITIR agent) is software that proactively retrieves and presents information based on a persons local context in an easily accessible yet nonintrusive manner. This paper describes three implemented JITIR agents: the Remembrance Agent, Margin Notes, and Jimminy. Theory and design lessons learned from these implementations are presented, drawing from behavioral psychology, information retrieval, and interface design. They are followed by evaluations and experimental results. The key lesson is that users of JITIR agents are not merely more efficient at retrieving information, but actually retrieve and use more information than they would with traditional search engines.

Margin notes: building a contextually aware associative memory

Both the Human Computer Interaction and Information Retrieval fields have developed techniques to allow a searcher to find the information they seek quickly. However, these techniques are designed to augment ones direct-recall memory, where the searcher is actively trying to find information. Associative memory, in contrast, happens automatically and continuously, triggering memories that relate to the observed world. This paper presents design techniques and heuristics for building “remembrance agents,” applications that watch a users context and proactively suggest information that may be of use. General design issues are discussed and illuminated by a description of Margin Notes, an automatic just-in-time information system for the Web.

Wearable computing meets ubiquitous computing: reaping the best of both worlds

This paper describes what we see as fundamental difficulties in both the pure ubiquitous computing and pure wearable computing paradigms when applied to context-aware applications. In particular, ubiquitous computing and smart room systems tend to have difficulties with privacy and personalization, while wearable systems have trouble with localized information, localized resource control, and resource management between multiple people. These difficulties are discussed and a peer-to-peer network of wearable and ubiquitous computing components is proposed as a solution. This solution is demonstrated through several implemented applications.

Wearable Computing and the Remembrance Agent

This paper gives an overview of the field of wearable computing. It covers the key differences between wearables and other portable computers, and discusses issues with the design and application for wearables. There then follows a specific example, the wearable remembrance agent — a proactive memory aid. The paper concludes with discussion of future directions for research and applications inspired by using the prototype.

PHISH Nets: planning heuristically in situated hybrid networks

Autonomous characters in interactive story systems are faced with challenges similar to those faced by both symbolic planners and situated-agent architectures. This thesis presents a new architecture based on the Agent Network Architecture developed by Pattie Maes. This new architecture, called PHISH-Nets, is especially well suited for the creation of characters with personality, who must satisfy multiple interacting goals in real-time in an only partially predictable dynamic environment. The algorithm has been used to implement The Big Bad Wolf, an autonomous character in a 3D graphical animated world based on The Three Little Pigs. The PHISH-Nets algorithm is explained in detail, and is informally verified in a series of experiments constructed within the graphical world. The results of the experiments are discussed, and limitations of the algorithms are presented along with possible future directions for this work. Thesis Supervisor: Pattie Maes Title: Associate Professor of Media Technology PHISH-Nets: Planning Heuristically In Situated Hybrid Networks by Bradley James Rhodes The following people served as readers for this thesis: in Reader Joseph Bates Professor of Computer Science Carnegie Mellon University School of Computer Science