Cesare Rocchi

Adaptive, intelligent presentation of information for the museum visitor in PEACH

The study of intelligent user interfaces and user modeling and adaptation is well suited for augmenting educational visits to museums. We have defined a novel integrated framework for museum visits and claim that such a framework is essential in such a vast domain that inherently implies complex interactivity. We found that it requires a significant investment in software and hardware infrastructure, design and implementation of intelligent interfaces, and a systematic and iterative evaluation of the design and functionality of user interfaces, involving actual visitors at every stage. We defined and built a suite of interactive and user-adaptive technologies for museum visitors, which was then evaluated at the Buonconsiglio Castle in Trento, Italy: (1) animated agents that help motivate visitors and focus their attention when necessary, (2) automatically generated, adaptive video documentaries on mobile devices, and (3) automatically generated post-visit summaries that reflect the individual interests of visitors as determined by their behavior and choices during their visit. These components are supported by underlying user modeling and inference mechanisms that allow for adaptivity and personalization. Novel software infrastructure allows for agent connectivity and fusion of multiple positioning data streams in the museum space. We conducted several experiments, focusing on various aspects of PEACH. In one, conducted with 110 visitors, we found evidence that even older users are comfortable interacting with a major component of the system.

The museum visit: generating seamless personalized presentations on multiple devices

The issue of the seamless interleaving of interaction with a mobile device and stationary devices is addressed, in a typical situation of educational entertainment: the visit to a museum. Some of the salient elements of the described work are the emphasis on multimodality in the dynamic presentation and coherence throughout the visit.The adopted metaphor is of a kind of contextualized TV-like presentation, useful for engaging (young) visitors. On the mobile device, personal video clips are dynamically generated from personalized verbal presentations; on larger stationary screens distributed throughout the museum, further background material and additional information is provided. A virtual presenter follows the visitors in their experience and gives advice on both types of devices and on the museum itself.

Automatic cinematography and multilingual NLG for generating video documentaries

Automatically constructing a complete documentary or educational film from scattered pieces of images and knowledge is a significant challenge. Even when this information is provided in an annotated format, the problems of ordering, structuring and animating sequences of images, and producing natural language descriptions that correspond to those images within multiple constraints, are each individually difficult tasks. This paper describes an approach for tackling these problems through a combination of rhetorical structures with narrative and film theory to produce movie-like visual animations from still images along with natural language generation techniques needed to produce text descriptions of what is being seen in the animations. The use of rhetorical structures from NLG is used to integrate separate components for video creation and script generation. We further describe an implementation, named Glamour, that produces actual, short video documentaries, focusing on a cultural heritage domain, and that have been evaluated by professional filmmakers.

A co-located interface for narration to support reconciliation in a conflict: initial results from Jewish and Palestinian youth

So called intractable conflicts may benefit from more modest and socially oriented approaches than those based on classical conflict resolution techniques. This paper is inspired by theories on small group intervention in a conflict. The general claim is that participants may achieve a greater understanding of and appreciation for the others viewpoint under conditions that support partaking in a tangible joint task and creating a shared narration. Our goal was to design a methodology wherein the extent to which technology contributes to conflict negotiation and resolution could be assessed. Specifically, a co-located interface for producing a joint narration as a tool for favouring reconciliation is presented and discussed. The results of an initial set of studies where the interface was used by Arab and Jewish youth in Israel provided insight into the usability of the various components of the technology and of the paradigm.

The design of a collaborative interface for narration to support reconciliation in a conflict

This paper is about the development of a face-to-face collaborative technology to support shifting attitudes of participants in conflict via a narration task. The work is based on two cultural elements: conflict resolution theory and the design of a collaboration enforcing interface designed specifically for the task. The general claim is that participants may achieve a greater understanding of and appreciation for the other’s viewpoint under conditions that support partaking in a tangible joint task and creating a shared narration. Specifically, a co-located interface for producing a joint narration as a tool for favoring reconciliation is presented and discussed. The process based on this technology implicitly includes classical steps in conflict resolution approaches, such as escalation and de-escalation. Our goal is to show that this interface is effective and constitutes an alternative to a typical face-to-face moderated discussion.

Non-intrusive user modeling for a multimedia museum visitors guide system

A personalized multimedia museum visitors guide system may be a valuable tool for improving user satisfaction in a museum visit. Personalization poses challenges to user modeling in the museum environment, especially when several different applications are supported by the same user model, where it is required to operate in a non-intrusive manner. This work presents the PEACH experience of non-intrusive user modeling supporting online dynamic multimedia presentation production and additional applications such as visit summary report generation.

Designing and redesigning an affective interface for an adaptive museum guide

The ideal museum guide should observe the user affective reactions to the presentations and adapt its behavior. In this paper we describe the user-centred design of an adaptive multimedia mobile guide with an affective interface. The novel approach has required a series of redesign cycles. We comment in particular on the last experiments we did with the prototype, users’ observations during interviews and more objective considerations based on logs. We show how the last design is better understood by the user, though there is still room for improvements.

An Agent-Based Architecture for Museum Visitors’ Guide Systems

Recent developments in museum visitors’ guides focus on context awareness, personalization and multimodal and multimedia information presentation to individuals and groups of visitors. However, the modern museum is becoming an “Active Museum”, which is a special example of an active environment that interacts with its inhabitants. Since recent museum visitors’ guides have focused more on the application and less on the system architecture and infrastructure, much effort is now being invested in the preparation of infrastructure that will support the specific research application. This work focuses on the architecture of the “Active Museum” as demonstrated by two research projects on museum visitors’ guides, and suggests a generic, layered architecture for such systems. Such architecture would facilitate research cooperation and increase its effectiveness, and also serve later as a basis for the development of museum visitors’ guides.

Rhetorical Patterns for Adaptive Video Documentaries

In this paper, we introduce an approach to the adaptive composition of video documentaries. The adaptation is based on templates that encode rules for the dynamic selection, sequencing and composition of video shots. We introduce a template language to define adaptation rules. Finally, we discuss rhetorical patterns, strategies that we abstracted out during the realization of a museum mobile guide.

Automatic video composition

This paper introduces an approach to generate video clips starting from an annotated commentary. The novelty of the approach lies in the use of rhetorical structure of the accompanying audio commentary in planning the video. The basic notions of cinematography are briefly introduced together with the Rhetorical Relation Theory to model the structure of a discourse. Then, the architecture of a video engine to automatically build video clips from the audio commentary annotated with respect to rhetorical relations is described. Finally, an application for a multimedia mobile guide in a museum is described.