Rui Nóbrega

A multi-touch tabletop for robust multimedia interaction in museums

The introduction of interaction technology in museum settings requires special care in different aspects including the relations with the different participants (public, artists, and curators), and the ability of the technology to provide rewarding experiences over extended periods of time for a demanding audience. This paper describes a hardware and software multi-touch table deployed in a contemporary art exhibition with many visitors. The design requirements are discussed and presented along with the development process. The paper also focuses on the different interaction and navigation mechanisms available and how they are used by the visitors and participants.

Flood Emergency Interaction and Visualization System

In this paper we describe a visualization system for an emergency simulation. We start by presenting a flooding emergency case scenario and all the elements that are involved in it. Then we describe the design decisions that were made in order to create a credible representation of the scene. This includes using a game engine to render the scenario using a three-dimensional terrain, with objects and information retrieved from geographic information systems. Additionally we describe experiments with new touch and tangible devices that support a war table like interaction with the simulation. We further describe our ideas for the emergency interface and conclude describing the directions for future work.

Magnetic augmented reality: virtual objects in your space

This paper proposes a framework to support interaction with virtual objects integrated in a real life scenario. In the proposed interaction concept, the user can reshape or re-design a real space using virtual objects using several pictures of the desired space. The images are analyzed for known features such as surfaces, floor or room orientation. Using these elements, it is possible to devise an augmented reality system where the user can add virtual objects to the scenario that react to its content. These are magnetic objects because they attach itself to elements on the scene such as floor or walls. This approach liberates the user from moving objects in three-dimensional space. Instead they just push around objects in a more natural interface. The paper studies the impact of adding these magnetism effects in the interaction with an augmented reality-editing interface.

Design your room: adding virtual objects to a real indoor scenario

This paper presents an interior design application that uses photos from real indoor spaces and rooms as input. The user is encouraged to take a picture of a certain place and virtually reshape it with furniture, different colors, textures and materials. The system analyses the still image of the physical space through image processing and computer vision algorithms to detect the world orientation relative to the camera. Knowing this, it is possible to create a natural interface where objects are laid on the floor and pushed around as in real life.

Dynamic Insertion of Virtual Objects in Photographs

Introducing virtual objects in photographs or video sequences presents several challenges, such as the pose estimation and the visually correct interaction boundaries of such objects. In this article a framework for the introduction of virtual objects in user-captured photos is discussed. Furthermore, the introduced virtual objects should be interactive and respond to real physical environments. The proposed detection system is semi-automatic and thus depends on the user to obtain the elements it needs. This operation should be significantly simple to accommodate the needs of a non-expert user. The system analyses a photo taken by the user and detects high-level features such as vanishing points, floor and scene orientation. Using these features it will be possible to create virtual mixed and augmented reality applications where the user takes one or more photos of a certain place and interactively introduces virtual objects or elements that blend with the picture in real time. This article discusses the techniques required to acquire images and information about the scenario involving the user. To demonstrate the framework, a proof-of-concept implementation is presented. This implementation was used to conduct a user study regarding the evaluation of the reliability of the concept. The presented results show a high reliability in the scene detection and that users are able and motivated to use this type of systems.

Navigation in past museum exhibitions using multimedia archives

Many exhibitions that occurred in the past in art museums left a large amount of records (pictures, videos and texts), which are distributed over several documents or places. This paper proposes a multimedia system to reconstruct the heritage and space from such exhibitions. The virtual reconstruction is based on collecting and recovering the documents and making them accessible using an integrated interface. The evolution of image and photograph visualization allows the virtual reconstruction of physical spaces, offering a new dimension of interaction with the user. Through image analysis techniques, the spatial relation between each photo in the archive is accessed allowing the semi-automatic construction of a virtual artwork exhibition.

Temporal and Spatial Evolution through Images

Image matching algorithms are used in image search, classification and retrieval but are also useful to show how urban structures evolve over time. Images have the power to illustrate and evoke past events and can be used to show the evolution of structures such as buildings and other elements present in the urban landscape. The paper describes a process and a tool to provide a chronological journey through time, given a set of photographs from different time periods. The developed tool provides the ability to generate visualizations of a geographic location, given a set of related images, taken at different periods in time. It automatically processes comparisons of images and establishes relationships between them. It also offers a semi-automated method to define relationships between parts of images.

Smart interface for reshaping photos in 3D

In this work we present an interactive prototype for an interface that supports the interaction with virtual objects integrated in a real life scenario. The user can reshape or re-design a real space with virtual objects using several pictures of the desired space. The images are analyzed for known features such as surfaces, edges, floor location and room orientation. Using these elements, it is possible to devise an augmented reality system where the user can add virtual objects to the scenario. The smart interface attaches the objects to the scene elements (e.g., floor) automatically. The current demo loads images from user files or takes a snapshot directly from the camera. The high-level features are automatically detected, but can be manually adjusted. A use-case example of an augmented reality application is presented.

Interactive Insertion of Virtual Objects in Photos and Videos

The introduction of virtual objects in photos or videos has been the focus of many Augmented Reality applications. This paper proposes a framework using image analysis methods to automatically detect scene features to introduce virtual 3D geometry objects. The current high-level features include surfaces, depth and scene orientation automatic detection. The main use of this technology is in AR tools, games or applications, which require the user to introduce an object that blends with a photographed or filmed scene. The main advantage of the proposed approach is that it can work only with one or two photos without prior knowledge, being ideal for mobile applications with camera or to be used with photo albums. Additional sensors can be added to increase reliability such as depth sensors or accelerometers but they are not essential. The main algorithms are intended to create a scene model in a few seconds and allow an interactive behaviour of the augmented objects.