Mario Nagamura

An experience using X3D for virtual cultural heritage

In this paper we present our experience in using Virtual Reality Technologies to accurately reconstruct and further explore ancient and historic city buildings. Virtual reality techniques provide a powerful set of tools to explore and access the history of a city. In order to explore, visualize and hear such history, we divided the process in three phases: historical data gathering and analysis; 3D reconstruction and modeling; interactive immersive visualization, auralization and display. The set of guidelines devised helped to put into practice the extensible tools available in VR but not always easy to put together by inexperienced users. These guidelines also helped the smoothness of our work and helped avoiding problems in the subsequent phases. Most importantly, the X3D standard provided an environment capable of helping the design and validation process as well as the visualization phase. To finalize, we present the results achieved and further analyze the extensibility of the framework. Although VR tools and techniques are widely available at present, there is still a gap between using the tools and really taking advantage of VR in historic architectural reconstruction so that users might immerse themselves into this world and thus be able to consider various scenarios and possibilities that might lead to new insightful inspiration. This is an ongoing process that we think will increase and help current architectural development.

Crosscale: A 3D virtual musical instrument interface

Inspired by principles for designing musical instruments we implemented a new 3D virtual instrument with a particular mapping of touchable virtual spheres to notes and chords of a given musical scale. The objects are metaphors for note keys organized in multiple lines, forming a playable spatial instrument where the player can perform certain sequences of notes and chords across the scale employing short gestures that minimizes jump distances. The idea of different arrangements for notes over the playable space is not new, being pursued on alternative keyboards for instance. This implementation employed an Oculus Rift and a Razer Hydra for gesture input and showed that customization of instrumental mappings using 3D tools can contribute to ease the performance of complex songs by allowing fast execution of specific note combinations.

OrbeVR: a handheld convex spherical virtual reality display

We present OrbeVR, a handheld concave spherical perspective-corrected display. OrbeVR displays combined images projected by multiple calibrated high-performance laser pico-projectors positioned inside a translucent sphere. Users position and OrbeVR are tracked, so the spherical display renders head-coupled perspectives with stereoscopic depth cues. OrbeVR is an extremely compact, lightweight and small Virtual Reality spherical display based on multiprojection technology. This emerging Virtual Reality technology enables exciting interactive display devices comparable to snow-globes.

3D stereoscopic game development technique for smart TVs

A 3D stereoscopic high definition game technique developed for Smart television sets is presented. Despite its limited processing power and low memory, the technique takes advantage of embedded television set streaming feature and user-controlled 2D objects. In order to test the technique, we developed a game for a commercial television. As a result, a new technique and an immersive challenging 3D stereoscopic racing game for Interactive Smart TVs are presented.

Batmen beyond: Natural 3D manipulation with the BatWand

In this work we present an interactive 3D object manipulation system using off the shelf mobile devices coupled with Augmented Reality (AR) technology that allows editing 3D objects by way of natural interactions based on tangible interfaces paradigms. The set-up consists of a mobile device, an interactive wand marker and AR markers laid on a table. The system allows users to change viewpoint and execute operations on 3D objects - simultaneous translation and rotation, scaling, cloning or deleting - by unconstrained natural interactions, leveraging users proficiency on daily object manipulation tasks and speeding up such typical 3D manipulation operations. Depth perception was significantly enhanced with dynamic shadows, allowing fast alignment and accurate positioning of objects. The prototype presented here allows successful completion of the three challenges proposed by the 2017 3DUI Contest, as validated by a preliminary informal user study with participants from the target audience and also from the general public.

ArcheoVR: Exploring Itapeva's archeological site

This demo presents a fully immersive and interactive virtual environment (VE) — the ArcheoVR, which represents Itapeva Rocky Shelter, a prehistoric archeological site in Brazil. W workflow started with a real world data capture — laser scanners, drones and photogrammetry. Captured information was transformed into a carefully designed realistic 3D scene and interactive features that allows users to experience the virtual archeological site in real-time. The main objective of this VR model is to allow the general public to feel and explore an otherwise restricted and ephemeral site and to assess prototype tools intended for future digital archaeological exploration.

Batmen — Hybrid collaborative object manipulation using mobile devices

In this work we present an interactive and collaborative 3D object manipulation system using the shelf mobile devices coupled with Augmented Reality (AR) technology that allows multiple users to collaborate concurrently on a scene. Each user interested in participating in this collaboration uses both a mobile device running android and a desktop (or laptop) working in tandem. The 3D scene was visualized by the user in the desktop system. The changes in the scene viewpoint changes and the object manipulations were performed using a mobile device through the AR. The system leverages users knowledge of common tasks performed on current mobile devices such as pinching for zooming in and out; swiping with one or two fingers for object rotation and press-and-hold for 2 seconds for object translation. As you will see in this video, we built a prototype system (in a maze style) and applied an informal user study with three experienced VR researchers. Users had to carry a 3D cube through three square rings along the maze. In resume, we diagnosed that working in a collaborative way (users A and B) was better and easier than individual one (user C). We registered more than 2 minutes late for the individual experience comparing to the teamwork. It may happen because the two player team shared information, functions and had a multi-perspective view during the task.