Michael Kruppa

Adaptive, intelligent presentation of information for the museum visitor in PEACH

The study of intelligent user interfaces and user modeling and adaptation is well suited for augmenting educational visits to museums. We have defined a novel integrated framework for museum visits and claim that such a framework is essential in such a vast domain that inherently implies complex interactivity. We found that it requires a significant investment in software and hardware infrastructure, design and implementation of intelligent interfaces, and a systematic and iterative evaluation of the design and functionality of user interfaces, involving actual visitors at every stage. We defined and built a suite of interactive and user-adaptive technologies for museum visitors, which was then evaluated at the Buonconsiglio Castle in Trento, Italy: (1) animated agents that help motivate visitors and focus their attention when necessary, (2) automatically generated, adaptive video documentaries on mobile devices, and (3) automatically generated post-visit summaries that reflect the individual interests of visitors as determined by their behavior and choices during their visit. These components are supported by underlying user modeling and inference mechanisms that allow for adaptivity and personalization. Novel software infrastructure allows for agent connectivity and fusion of multiple positioning data streams in the museum space. We conducted several experiments, focusing on various aspects of PEACH. In one, conducted with 110 visitors, we found evidence that even older users are comfortable interacting with a major component of the system.

The museum visit: generating seamless personalized presentations on multiple devices

The issue of the seamless interleaving of interaction with a mobile device and stationary devices is addressed, in a typical situation of educational entertainment: the visit to a museum. Some of the salient elements of the described work are the emphasis on multimodality in the dynamic presentation and coherence throughout the visit.The adopted metaphor is of a kind of contextualized TV-like presentation, useful for engaging (young) visitors. On the mobile device, personal video clips are dynamically generated from personalized verbal presentations; on larger stationary screens distributed throughout the museum, further background material and additional information is provided. A virtual presenter follows the visitors in their experience and gives advice on both types of devices and on the museum itself.

Adaptive mobile guides

In this chapter we discuss various aspects of adaptive mobile guide applications. After having motivated the need for web based mobile applications, we will discuss technologies that are needed to enable adaptive mobile web applications, including not only positioning technologies but also sensor technologies needed to determine additional information on the context and situation of usage. We will also address issues of modeling context and situations before giving an overview on existing systems coming from three important classes of mobile guides: museum guides, navigation systems and shopping assistants. The chapter closes with an extensive discussion of relevant attributes of web based mobile guides.

Automated Usability Evaluation during Model-Based Interactive System Development

In this paper we describe an approach to efficiently evaluate the usability of an interactive application that has been realized to support various platforms and modalities. Therefore we combine our Multi-Access Service Platform (MASP), a model-based runtime environment to offer multimodal user interfaces with the MeMo workbench which is a tool supporting an automated usability analysis. Instead of deriving a system model by reverse-engineering or annotating screenshots for the automated usability analysis, we use the semantics of the runtime models of the MASP. This allows us to reduce the evaluation effort by automating parts of the testing process for various combinations of platforms and user groups that should be addressed by the application. Furthermore, by testing the application at runtime, the usability evaluation can also consider system dynamics and information that are unavailable at design time.

The virtual room inhabitant: intuitive interaction with intelligent environments

In this paper we describe a new way to improve the usability of complex hardware setups in Intelligent Environments. By introducing a virtual character, we facilitate intuitive interaction with our Intelligent Environment. The character is capable of freely moving along the walls of the room. The character is aware of the users position and orientation within the room. In this way, it may offer situated assistance as well as unambiguous references to physical objects by means of combined gestures, speech and physical locomotion. We make use of a steerable projector and a spatial audio system, in order to position the character within the environment.

Performing physical object references with migrating virtual characters

In this paper we address the problem of performing references to wall mounted physical objects. The concept behind our solution is based on virtual characters. These characters are capable of performing reasonable combinations of motion, gestures and speech in order to disambiguate references to real world objects. The new idea of our work is to allow characters to migrate between displays to find an optimal position for the reference task. We have developed a rule-based system that, depending on the individual situation in which the reference is performed, determines the most appropriate reference method and technology from a number of different alternatives. The described technology has been integrated in a museum guide prototype combining mobile- and stationary devices.

Looking for unexpected consequences of interface design decisions: the MeMo workbench

This paper discusses and illustrates work in progress on the MEMO workbench for early model-based usability evaluation of interface designs. Characteristic features of the workbench include (a) the prediction of errors via rules that refer to user attributes; and (b) the automatic generation of methods for performing specific tasks and for recovering from errors.

The Shopping Experience of Tomorrow: Human-Centered and Resource-Adaptive

What would the shopping experience of tomorrow look like? In this chapter we propose several human-centered and resource-adaptive ideas to this question. Throughout the whole chapter we explain our ideas with the recurrent theme of a shop that consists of instrumented shelves, public displays, audio systems, and mobile devices for each user.

Rule-Based Approach for Simulating Age-Related Usability Problems

Ambient Assisted Living requires easy to use interfaces, making usability a critical feature. Because usability evaluations are resource and time consuming, several automation efforts have been made, one of which is the simulation of users interacting with UIs. In this article, we present ongoing work of a tool for automated usability simulations that allows simulating agerelated deficits. The tool is specifically intended to be used by IT practitioners, i.e. in difference to cognitive architectures that allow similar simulations, this tool does not require extensive knowledge in cognitive science. A core component of the simulation tool is its rule-based User Model (UM). During a simulated interaction, the UM selects actions causing a model of the UI to change states until a specified task goal is satisfied or the UM “gives up”. Interactions of the UM are calculated from probabilities which are informed by rules drawing on user and UI attributes. Using a Monte Carlo approach, the simulation is iterated, resulting in a set of task solutions where non-optimal solutions may indicate usability problems. By analyzing which rules led the UM to interact non-optimally, our approach can offer hints on how to improve the UI. While our approach cannot render user-based evaluations unnecessary, our aim is to substantially reduce the effort involved in usability testing of UIs as well as to provide an automated tool that can be used early on in the development process.

A Pool of Topics: Interactive Relational Topic Visualization for Information Discovery

In this paper, we present a novel relational visualization that supports people at information discovery tasks in digital libraries. This visualization displays search query results structured into topics and highlights the intersections between them. The proposed visual representation introduces interactive drag-and-drop operations for manipulation of the generated topics. These operations mirror the human online searching strategies that involve boolean AND, OR, and NOT operators. In doing so, the information seeker can refine (or relax) a search query in an interactive way during a focusing or a defocusing phase. The intersections of topics are made explicitly visible to enable the information seeker to avoid frustrating “no hits” situations.