Holger Schnädelbach

Expected, sensed, and desired: A framework for designing sensing-based interaction

Movements of interfaces can be analyzed in terms of whether they are expected, sensed, and desired. Expected movements are those that users naturally perform; sensed are those that can be measured by a computer; and desired movements are those that are required by a given application. We show how a systematic comparison of expected, sensed, and desired movements, especially with regard to how they do not precisely overlap, can reveal potential problems with an interface and also inspire new features. We describe how this approach has been applied to the design of three interfaces: pointing flashlights at walls and posters in order to play sounds; the Augurscope II, a mobile augmented reality interface for outdoors; and the Drift Table, an item of furniture that uses load sensing to control the display of aerial photographs. We propose that this approach can help to build a bridge between the analytic and inspirational approaches to design and can help designers meet the challenges raised by a diversification of sensing technologies and interface forms, increased mobility, and an emerging focus on technologies for everyday life.

The augurscope: a mixed reality interface for outdoors

The augurscope is a portable mixed reality interface for outdoors. A tripod-mounted display is wheeled to different locations and rotated and tilted to view a virtual environment that is aligned with the physical background. Video from an onboard camera is embedded into this virtual environment. Our design encompasses physical form, interaction and the combination of a GPS receiver, electronic compass, accelerometer and rotary encoder for tracking. An initial application involves the public exploring a medieval castle from the site of its modern replacement. Analysis of use reveals problems with lighting, movement and relating virtual and physical viewpoints, and shows how environmental factors and physical form affect interaction. We suggest that problems might be accommodated by carefully constructing virtual and physical content

Camping in the digital wilderness: tents and flashlights as interfaces to virtual worlds

A projection screen in the shape of a tent provides children with a shared immersive experience of a virtual world based on the metaphor of camping. RFID aerials at its entrances sense tagged children and objects as they enter and leave. Video tracking allows multiple flashlights to be used as pointing devices. The tent is an example of a traversable interface, designed for deployment in public spaces such as museums, galleries and classrooms.

Traversable interfaces between real and virtual worlds

Traversable interfaces establish the illusion that virtual and physical worlds are joined together and that users can physically cross from one to the other. Our design for a traversable interface combines work on tele-embodiment, mixed reality boundaries and virtual environments. It also exploits non-solid projection surfaces, of which we describe four examples. Our design accommodates the perspectives of users who traverse the interface and also observers who are present in the connected physical and virtual worlds, an important consideration for performance and entertainment applications. A demonstrator supports encounters between members of our laboratory and remote visitors.

Performing thrill: designing telemetry systems and spectator interfaces for amusement rides

Fairground: Thrill Laboratory was a series of live events that augmented the experience of amusement rides. A wearable telemetry system captured video, audio, heart-rate and acceleration data, streaming them live to spectator interfaces and a watching audience. In this paper, we present a study of this event, which draws on video recordings and post-event interviews, and which highlights the experiences of riders, spectators and ride operators. Our study shows how the telemetry system transformed riders into performers, spectators into an audience, and how the role of ride operator began to include aspects of orchestration, with the relationship between all three roles also transformed. Critically, the introduction of a telemetry system seems to have had the potential to re-connect riders/performers back to operators/orchestrators and spectators/audience, re-introducing a closer relationship that used to be available with smaller rides. Introducing telemetry to a real-world situation also creates significant complexity, which we illustrate by focussing on a moment of perceived crisis.

Citywide: Supporting Interactive Digital Experiences Across Physical Space

Abstract: The Citywide project is exploring ways in which technology can provide people with rich and engaging digital experiences as they move through physical space, including historical experiences, performances and games. This paper describes some initial results and experiences with this project based upon two prototype demonstrators. In the first, we describe an application in which a search party explores an archaeological site, uncovering enacted scenes within the virtual world that are of a historical relevance to their particular physical location. In the second, we describe a museum experience where participants explore an outdoors location, hunting for buried virtual artifacts that they then bring back to a museum for a more detailed study. Our demonstrators employ a varied set of devices, including mobile wireless interfaces for locating hotspots of virtual activity when outdoors, to give different experiences of the virtual world depending upon location, task, available equipment and accuracy of tracking. We conclude by discussing some of the potential advantages of using an underlying shared virtual world to support interactive experiences across extended physical settings.

ExoBuilding: Physiologically Driven Adaptive Architecture

Our surroundings are becoming infused with sensors measuring a variety of data streams about the environment, people and objects. Such data can be used to make the spaces that we inhabit responsive and interactive. Personal data in its different forms are one important data stream that such spaces are designed to respond to. In turn, one stream of personal data currently attracting high levels of interest in the HCI community is physiological data (e.g., heart rate, electrodermal activity), but this has seen little consideration in building architecture or the design of responsive environments. In this context, we developed a prototype mapping a single occupant’s respiration to its size and form, while it also sonifies their heartbeat. The result is a breathing building prototype, formative trials of which suggested that it triggers behavioral and physiological adaptations in inhabitants without giving them instructions and it is perceived as a relaxing experience. In this paper, we present and discuss the results of a controlled study of this prototype, comparing three conditions: the static prototype, regular movement and sonification and a biofeedback condition, where the occupant’s physiological data directly drives the prototype and presents this data back to them. The study confirmed that the biofeedback condition does indeed trigger behavioral changes and changes in participants’ physiology, resulting in lower respiration rates as well as higher respiration amplitudes, respiration to heart rate coherence and lower frequency heart rate variability. Self-reported state of relaxation is more dependent on inhabitant preferences, their knowledge of physiological data and whether they found space to ‘let go’. We conclude with a discussion of ExoBuilding as an immersive but also sharable biofeedback training interface and the wider potential of this approach to making buildings adapt to their inhabitants.

Moving office: inhabiting a dynamic building

Mixed Reality Architecture (MRA) supports distributed teams in their everyday work activities by linking multiple physical spaces across a shared three-dimensional virtual world. User configurable audio-visual connections give the inhabitants of MRA full control over whom they want to be in contact with and when they make themselves available, as well as over the overall configuration. We report on the design of MRA, its deployment in an office environment and results from a long-term observational study. The study shows that MRA supports the management of awareness, social interaction and privacy well, that the architectural design features are crucial for this process and that the dynamic architectural topology of MRA and social interaction within it are linked in a fundamental way.

Tension Space Analysis: Exploring Community Requirements for Networked Urban Screens

This paper draws on the design process, implementation and early evaluation results of an urban screens network to highlight the tensions that emerge at the boundary between the technical and social aspects of design. While public interactive screens in urban spaces are widely researched, the newly emerging networks of such screens present fresh challenges. Researchers wishing to be led by a diverse user community may find that the priorities of some users, directly oppose the wishes of others. Previous literature suggests such tensions can be handled by ‘goal balancing’, where all requirements are reduced down to one set of essential, implementable attributes. Contrasting this, this paper’s contribution is ‘Tension Space Analysis’, which broadens and extends existing work on Design Tensions. It includes new domains, new representational methods and offers a view on how to best reflect conflicting community requirements in some aspects or features of the design.

Unearthing Virtual History: Using Diverse Interfaces to Reveal Hidden Virtual Worlds

We describe an application in which museum visitors hunt for virtual history outdoors, capture it, and bring it back indoors for detailed inspection. This application provides visitors with ubiquitous access to a parallel virtual world as they move through an extended physical space. Diverse devices, including mobile wireless interfaces for locating hotspots of virtual activity outdoors, provide radically different experiences of the virtual depending upon location, task, and available equipment. Initial reflections suggest that the physical design of such devices needs careful attention so as to encourage an appropriate style of use. We also consider the extension of our experience to support enacted scenes. Finally, we discuss potential benefits of using diverse devices to make a shared underlying virtual world ubiquitously available throughout physical space.