Thomas Holz

Using Mixed Reality Agents as Social Interfaces for Robots

Endowing robots with a social interface is often costly and difficult. Virtual characters on the other hand are comparatively cheap and well equipped but suffer from other difficulties, most notably their inability to interact with the physical world. This paper details our wearable solution to combining physical robots and virtual characters into a mixed reality agent (MiRA) through mixed reality visualisation. It describes a pilot study demonstrating our system, and showing how such a technique can offer a viable alternative cost effective approach to enabling a rich social interface for human-robot interaction.

Mixing robotic realities

This paper contests that Mixed Reality (MR) offers a potential solution in achieving transferability between Human Computer Interaction (HCI) and Human Robot Interaction (HRI). Virtual characters (possibly of a robotic genre) can offer highly expressive interfaces that are as convincing as a human, are comparably cheap and can be easily adapted and personalized. We introduce the notion of a mixed reality agent, i.e. an agent consisting of a physical robotic body and a virtual avatar displayed upon it. We realized an augmented reality interface with a Head-Mounted Display (HMD) in order to interact with such systems and conducted a pilot study to demonstrate the usefulness of mixed reality agents in human-robot collaborative tasks.

Social situated agents in virtual, real and mixed reality environments

This paper details a framework for explicit deliberative control of socially and physically situated agents in virtual, real and mixed reality environments. The objective is to blur the traditional boundaries between the real and the virtual and provide a standardized methodology for intelligent agent control specifically designed for social interaction. The architecture presented in this paper embraces the fusion between deliberative social reasoning mechanisms and explicit tangible behavioural mechanisms for human-agent social interaction.

Why, when and how to use augmented reality agents (AuRAs)

Over the last number of years, multiple research projects have begun to create augmented reality (AR) applications that use augmented reality agents, or AuRAs, as their principle interaction and development paradigm. This paper aims to address this new and distinct field of AuRAs by asking three questions: why should AuRAs be researched, when are they a useful paradigm, and how can they be developed? The first question explores the motivation behind applying AuRAs to AR. Specifically, it investigates whether AuRAs are purely an interaction paradigm, or whether they can also serve as a development paradigm, by outlining in which circumstances it is appropriate for a project to use AuRAs and where their addition would only add unnecessary complexity. A navigational experiment, performed in simulated AR, explores the second question of when AuRAs can be a useful concept in AR applications. Results from this experiment suggest that an embodied virtual character allows for faster navigation along a shorter route than directional arrows or marking the target with an AR “bubble”. An exploration of the limitations of the simulated AR environment illuminates how faithfully the experiment recreated the environmental challenges that AuRAs can help to address. Finally, the question of how to develop such applications is addressed through the introduction of the agent factory augmented reality toolkit that allows the rapid prototyping of such applications. Results from a usability study on the toolkit are also presented.

Facilitating ubiquitous interaction using intelligent agents

Facilitating intuitive interaction is a prerequisite for the ubiquitous computing paradigm in all its manifestations. How to achieve such interaction in practice remains an open question. Such interfaces must be perceived as being intuitive across a variety of contexts, including those of the hosting devices. Indeed, the heterogeneity of the device population raises significant challenges.While individual devices and the interaction modalities supported by, each satisfy the requirements of individual domains, integrating diverse devices such that the user experiences is perceived as consistent and intuitive is problematic. This chapter discusses and illustrates how intelligent agents may be harnessed for integrating a range of diverse interface and interaction modalities such that the ubiquitous user interface concept may be validated.

Mixed Reality Agent (MiRA) Chameleons

Human-Robot Interaction poses significant research challenges. Recent research suggests that rsonalisation and individualisation are key factors for establishing lifelong human-robot relationships. This raises difficulties as roboticists seek to incorporate robots into the digital society where an creasing amount of human activities relies on digital technologies and ubiquitous infrastructures. In essence, a robot may be perceived as either an embedded or mobile artefact in an arbitrary environment that must be interacted with in a seamless and intuitive fashion. This chapter explores some of the ternative ways and the design ssues to achieve these objectives. Specifically, it describes a new system, which we call Mixed Reality Agent (MiRA) Chameleon, that combines the latest advancements on agent-based ubiquitous architectures with mixed reality technology to deliver personalised and ubiquitous robot agents.