Fabio Pittarello

Semantic description of 3D environments: a proposal based on web standards

While the number of virtual environments available on the net is constantly increasing, most of them are composed by a wide number of low-level geometric objects that lack any semantic description. Such situation prevents advanced uses of the data contained inside the environments, such as selection and extraction of semantic objects or advanced queries that refer to high-level properties of the environment. Recently some attempts have been performed in order to annotate 3D environments using the descriptive capabilities of MPEG-7 standard. While the solutions proposed are interesting, there are still a number of issues to be solved, including the definition of scene-independent ontologies that can be useful in different situations (e.g., 3D world validation, semantic search through a set of worlds, etc.).This work proposes an alternate approach for associating semantic information to 3D worlds based on the integration of two web standards: the X3D language and the semantic web. The approach is characterized also by the definition of scene-independent ontologies and by the definition of semantic zones that complement the role of semantic objects for giving a complete description of the environment.In order to show the potentialities of such approach the paper will illustrate an application scenario characterized by the extraction of the semantic information from an X3D document and the associated ontology for generating a high-level and multilevel textual description of a tour through the 3D environment described by them.

Observing and adapting user behavior in navigational 3D interfaces

In a navigation-oriented interaction paradigm, such as desktop, mixed and augmented virtual reality, recognizing the user needs is a valuable improvement, provided that the system is able to correctly anticipate the user actions. Methodologies for adapting both navigation and content allow the user to interact with a customized version of the 3D world, lessening the cognitive load needed for accomplishing tasks such as finding places and objects, and acting on virtual devices.This work discusses adaptivity of interaction in 3D environments, obtained through the coordinated use of three approaches: structured design of the interaction space, distinction between a base world layer and an interactive experience layer, and user monitoring in order to infer interaction patterns. Identification of such recurring patterns is used for anticipating users actions in approaching places and objects of each experience class. An agent based architecture is proposed, and a simple application related to consumer e-business is analyzed.

Accessing information through multimodal 3D environments: towards universal access

Abstract3D environments represent a great opportunity for universal access to information, as they offer an intuitive interaction paradigm, similar to what is experienced by humans in their everyday lives. In spite of that, several 3D interfaces are characterized by poor structures and are hard to navigate. This paper presents the multimodal concept of the Interaction Locus (IL) as a means to give structure to 3D scenes, helping the user to interact with and access information inside them. The concept was initially developed with particular reference to desktop virtual reality (2.5 D virtual reality), but it is general enough to be extended to other contexts, such as real 3D scenes. The final part of this work shows how the IL concept addresses the need for a unified authoring methodology, capable of allowing access to different target user groups from a variety of different devices.

ToBoA-3D: an architecture for managing top-down and bottom-up annotated 3D objects and spaces on the web

The association of a high-level semantics to the geometric components of 3D worlds for the net is gaining popularity, because it enables advanced uses of the 3D objects and it may improve their findability, extended to a network of annotated 3D worlds. Most of the proposals for annotating 3D worlds for the net are characterized by top-down approaches, where selected authors annotate 3D worlds in relation to predefined classes and relations. A different approach - introduced only in recent times for the 3D web but widely diffused in the hypertextual web - permits to users to annotate freely objects and spaces, originating bottom-up structures defined as folksonomies. This work introduces a web-based specification, a software architecture and a set of interfaces for annotating 3D worlds. The proposal is based on the X3D standard and enables both top-down and bottom-up annotation styles. The components of the software architecture - named ToBoA-3D - can be used in different scenarios, ranging from the complete management of a centralized set of annotated 3D worlds to the creation of a network of top-down annotated 3D worlds distributed on the net. Besides, the solution proposed can be applied to many X3D worlds already available on the net, without compromising the interaction possibilities implemented by the world designers. The architecture includes a search module that may use the annotations retrieved during the browsing phase for triggering the research of other objects and spaces that share the same labels but are part of other 3D worlds. The integration of navigation and searching - introduced in ToBoA-3D but unavailable for most of the 3D worlds for the net - represents an additional feature that may improve furthermore the findability of the resources. The paper defines also an enhanced model for defining the spatial limits of complex spaces, that is used as a basis for annotating these entities. The presentation of the user interfaces, applied to an example related to the architectural domain, concludes the work.

Navigation help in 3D worlds: some empirical evidences on use of sound

The concept of Interaction Locus (IL) has been introduced to help the users to orient, navigate, and identify relevant interaction areas in 3D Virtual Environments (VEs). The IL is a multimodal concept: it adds to the 3D visual scene parallel information channels that are perceived by other senses. In particular, the IL emphasizes the role of music as a navigation aid in a VE. This paper reports three user-evaluations of different IL enriched virtual worlds, and in particular of the role of the IL auditory component. Results suggest that audio in 3D plays not only an aesthetic role, which the users greatly appreciate, but also a functional role simplifying navigation and helping the users to recognise scenes in the environment. Such a functional role however is subordinated to a proper understanding of the link between music and virtual space. While these experiments refer to desktop virtual reality environments, their findings are general enough to inform the design of navigational tools for other segments of the mixed reality domain.

Designing a context-aware architecture for emotionally engaging mobile storytelling

This work illustrates the design of a context-aware software architecture supporting the narration of interactive stories for mobile users. The peculiarity of this work is the use of an extended set of context dimensions, including the surrounding environment and the user social network, for enhancing the engagement and the emotional impact on the users experiencing the story.

CASTOR: learning to create context-sensitive and emotionally engaging narrations in-situ

This paper describes the design and the experimentation of a tablet-based authoring system for supporting the creation of stories in-situ, in the context of a learning path that has led 19 children aged 7 of the second class of a Primary School to learn how to build stories using different techniques, ranging from the traditional writing techniques exercised in class to the in-situ authoring with the tablet application. The project was characterized by a strong involvement of the teachers, that modeled a substantial part of the ordinary classroom work for embedding the in-situ experimentation and designing a smooth learning path. To our knowledge CASTOR is one of the first authoring systems allowing the direct creation of structured stories in-situ, rather than the simple gathering of material. Its architecture is based on a story model presented in previous works [3, 20] and characterized by the use of the environmental and social context for augmenting the emotional engagement of the story listeners. All the components of the architecture have been designed for complying with the learning needs and the skills of a class of young children. The final part of the paper will present the results of the experimentation, that gave interesting insights not only on the use of the authoring interface itself, but also on its effects on the children learning process.

From real to metaphoric maps: Cartography as a visual language for organizing and sharing knowledge

In this paper we discuss about geographic representations as a basis for describing, organizing, accessing and understanding heterogeneous shared information on the web. Maps are popular on the web, because of the reference to space, the most important domain of human experience, the proliferation of location aware devices and services and the availability of a set of tools that enable an heterogeneous population of users to explore and even modify these representations. Metaphorical maps, representing concepts and relations of a specific knowledge domain with symbols taken from another well known and widely used domain, couple the benefits of cartographic representation with the power and intuitiveness of the metaphor, permitting the communication and sharing of such knowledge. We introduce a classification of maps based on antinomies between real and imaginary worlds, and between direct and metaphoric knowledge; we argue that cartography can be used as a visual language for organizing and sharing knowledge related to different semantic domains, supporting our arguments with examples. Finally, we define a set of functions and related data structures able to support a user in browsing cartographic representations using state-of-art tools and systems available on the web.

Querying and navigating a database of images with the magical objects of the wizard Zurlino

This work is part of a research targeted at experimenting the use of physical artifacts for the retrieval of multimedia information. Tangible interfaces --- that couple physical artifacts to digital data --- are described in different research works, and a number of studies focus on tangibles for children. In spite of that, most of the work done for the kids is related to gaming or learning. This work is focused on a less explored domain, that of the access to information systems. We propose a tangible interface for enabling preschoolers to query and navigate multimedia information. The approach was tested with a class of 27 preschoolers, that where engaged in a game with the magical objects of the wizard Zurlino. The experiment gave us interesting insights about the suitability of the system for preschool children, its ease of use and the need for support by educators.

Improving access of elderly people to real environments: a semantic based approach

Access to real environments is often conditioned by a number of issues, including the skills of the user (i.e. affected by aging, physical and psychological deficiencies, etc.) and the complexity of the real environment itself. This work proposes an approach for helping users with different skills, including elderly people, to navigate through complex real scenes; such approach is based on the semantic description of the objects and zones that characterize the environment itself and takes advantage of an implementation architecture based on web standards for generating navigational support. A case study related to the creation of a guided tour through the indoor and outdoor locations of the city of Venice, accessible through a multimodal web browser, is presented.