Bruno Rodrigues De Araújo

Snake Charmer: Physically Enabling Virtual Objects

Augmented and virtual reality have the potential of being indistinguishable from the real world. Holographic displays, including head mounted units, support this vision by creating rich stereoscopic scenes, with objects that appear to float in thin air - often within arms reach. However, one has but to reach out and grasp nothing but air to destroy the suspension of disbelief. Snake-charmer is an attempt to provide physical form to virtual objects by revisiting the concept of Robotic Graphics or Encountered-type Haptic interfaces with current commodity hardware. By means of a robotic arm, Snake-charmer brings physicality to a virtual scene and explores what it means to truly interact with an object. We go beyond texture and position simulation and explore what it means to have a physical presence inside a virtual scene. We demonstrate how to render surface characteristics beyond texture and position, including temperature; how to physically move objects; and how objects can physically interact with the users hand. We analyze our implementation, present the performance characteristics, and provide guidance for the construction of future physical renderers.

Special Section on Touching the 3rd Dimension: Mockup Builder: 3D modeling on and above the surface

We present Mockup Builder, a semi-immersive environment for conceptual design which allows virtual mockups to be created using gestures. Our goal is to provide familiar ways for people to conceive, create and manipulate three-dimensional shapes. To this end, we developed on-and-above-the-surface interaction techniques based on asymmetric bimanual interaction for creating and editing 3D models in a stereoscopic environment. Our approach combines both hand and finger tracking in the space on and above a multi-touch surface. This combination brings forth an alternative design environment where users can seamlessly switch between interacting on the surface or above it to leverage the benefit of both interaction spaces. A formal user evaluation conducted with experienced users shows very promising avenues for further work towards providing an alternative to current user interfaces for modeling.

Adaptive polygonization of implicit surfaces

We present an algorithm for polygonizing closed implicit surfaces, which produces meshes adapted to the local curvature of the surface. Our method is similar to, but not based on, Marching Triangles, in that we start from a point on the surface and develop a mesh from that point using a surface-tracking approach. However, our approach works by managing fronts, or sets of points on the border of the current polygonization. Fronts can subdivide to form new fronts or merge if they become adjacent. In a marked departure from previous approaches, our meshes approximate the surface through heuristics relying on curvature. Furthermore, our method works completely on-the-fly, resolving cracks as it proceeds, without the need for any post-remeshing step to correct failures. We have tested the algorithm with three different representations of implicit surfaces, variational, analytical and MPU, using non-trivial data sets, yielding results that illustrate the flexibility and scalability of our technique. Performance comparisons with variants of Marching Cubes show that our approach is capable of good accuracy and meshing quality without sacrificing computing resources.

Mid-air interactions above stereoscopic interactive tables

Stereoscopic tabletops offer unique visualization capabilities, enabling users to perceive virtual objects as if they were lying above the surface. While allowing virtual objects to coexist with user actions in the physical world, interaction with these virtual objects above the surface presents interesting challenges. In this paper, we aim to understand which approaches to 3D virtual object manipulations are suited to this scenario. To this end, we implemented five different techniques based on the literature. Four are mid-air techniques, while the remainder relies on multi-touch gestures, which act as a baseline. Our setup combines affordable non-intrusive tracking technologies with a multi-touch stereo tabletop, providing head and hands tracking, to improve both depth perception and seamless interactions above the table. We conducted a user evaluation to find out which technique appealed most to participants. Results suggest that mid-air interactions, combining direct manipulation with six degrees of freedom for the dominant hand, are both more satisfying and efficient than the alternatives tested.

Object-Oriented Drawing

We present Object-Oriented Drawing, which replaces most WIMP UI with Attribute Objects. Attribute Objects embody the attributes of digital content as UI objects that can be manipulated through direct touch gestures. In the paper, the fundamental UI concepts are presented, including Attribute Objects, which may be moved, cloned, linked, and freely associated with drawing objects. Other functionalities, such as attribute-level blending and undo, are also demonstrated. We developed a drawing application based on the presented concepts with simultaneous touch and pen input. An expert assessment of our application shows that direct physical manipulation of Attribute Objects enables a user to quickly perform interactions which were previously tedious, or even impossible, with a coherent and consistent interaction experience throughout the entire interface.

Combining bimanual manipulation and pen-based input for 3D modelling

Multitouch enabled surfaces can bring advantages to modelling scenarios, in particular if bimanual and pen input can be combined. In this work, we assess the suitability of multitouch interfaces to 3D sketching tasks. We developed a multitouch enabled version of ShapeShop, whereby bimanual gestures allow users to explore the canvas through camera operations while using a pen to sketch. This provides a comfortable setting familiar to most users. Our contribution focuses on comparing the combined approach (bimanual and pen) to the pen-only interface for similar tasks. We conducted the evaluation helped by ten sketching experts who exercised both techniques. Results show that our approach both simplifies workflow and lowers task times, when compared to the pen-only interface, which is what most current sketching applications provide.

Multi-Device Storyboards for Cinematic Narratives in VR

Virtual Reality (VR) narratives have the unprecedented potential to connect with an audience through presence, placing viewers within the narrative. The onset of consumer VR has resulted in an explosion of interest in immersive storytelling. Planning narratives for VR, however, is a grand challenge due to its unique affordances, its evolving cinematic vocabulary, and most importantly the lack of supporting tools to explore the creative process in VR. In this paper, we distill key considerations with the planning process for VR stories, collected through a formative study conducted with film industry professionals. Based on these insights we propose a workflow, specific to the needs of professionals creating storyboards for VR film, and present a multi-device (tablet and head-mounted display) storyboard tool supporting this workflow. We discuss our design and report on feedback received from interviews following demonstration of our tool to VR film professionals.

Hammer Time!: A Low-Cost, High Precision, High Accuracy Tool to Measure the Latency of Touchscreen Devices

We report on the Latency Hammer, a low-cost yet highaccuracy and high-precision automated tool that measures the interface latency of touchscreen devices. The Hammer directly measures latency by triggering a capacitive touch event on a device using an electrically actuated touch simulator, and a photo sensor to monitor the screen for a visual response. This allows us to measure the full end-toend latency of a touchscreen system exactly as it would be experienced by a user. The Hammer does not require human interaction to perform a measurement, enabling the acquisition of large datasets. We present the operating principles of the Hammer, and discuss its design and construction; full design documents are available online. We also present a series of tools and equipment that were built to assess and validate the performance of the Hammer, and demonstrate that it provides reliable latency measurements.

3D modelling of laser scanned and photogrammetric data for digital documentation: the Mosteiro da Batalha case study

Advances in both terrestrial laser scanning hardware and photogrammetric systems combined are creating increasingly precise and rich 3D coloured data. In this article we show how computer graphics and visualization techniques have played an important role in real-time visualization, data management, modelling, and data fusion in an increasing number of applications such as surveying engineering, structure analysis, architecture, archaeology and cultural heritage. Specifically, we describe the typical modelling steps involved in the creation of a range of digital documents provided by the 3D digitization company Artescan to customers. We present how these modelling steps were applied in the context of creating digital documents used in the preservation of Mosteiro da Batalha.

IMPROVE: collaborative design review in mobile mixed reality

In this paper we introduce an innovative application designed tomake collaborative design review in the architectural and automotive domainmore effective. For this purpose we present a system architecture whichcombines variety of visualization displays such as high resolution multitile displays, TabletPCs and head-mounted displays with innovative 2D and 3DInteraction Paradigms to better support collaborative mobile mixed realitydesign reviews. Our research and development is motivated by two usescenarios: automotive and architectural design review involving real users fromPage\Park architects and FIAT Elasis. Our activities are supported by the EUIST project IMPROVE aimed at developing advanced display techniques,fostering activities in the areas of: optical see-through HMD development usingunique OLED technology, marker-less optical tracking, mixed reality rendering,image calibration for large tiled displays, collaborative tablet-based andprojection wall oriented interaction and stereoscopic video streaming for mobileusers. The paper gives an overview of the hardware and software developmentswithin IMPROVE and concludes with results from first user tests.