João Paulo Silva do Monte Lima

Photorealistic rendering for Augmented Reality: A global illumination and BRDF solution

This paper presents a solution for the photorealistic rendering of synthetic objects into dynamic real scenes, in Augmented Reality applications. In order to achieve this goal, an Image Based Lighting approach is used, where environment maps with different levels of glossiness are generated for each virtual object in the scene at every frame. Due to this, illumination effects, such as color bleeding and specular reflections, can be simulated for virtual objects in a consistent way. A unifying sampling method for the spherical harmonics transformation pass is also used. It is independent of map format and does not need to apply different weights for each sample. The developed technique is combined with an extended version of Lafortune Spatial BRDF, featuring Fresnel effect and an innovative tangent rotation parameterization. The solution is evaluated in various Augmented Reality case studies, where other features like shadowing and lens effects are also exploited.

Virtual Reality in Brazil 2011: RPR-SORS: Real-time photorealistic rendering of synthetic objects into real scenes

This paper presents a review of the Photorealistic Augmented Reality field and proposes a solution for interactively rendering virtual objects into dynamic real scenes in a photorealistic way. This solution features a rendering pipeline that comprises techniques regarding illumination, reflectance model, shadowing, composition, and camera effects. The techniques are chained in a flexible way, allowing the user to choose which techniques are to be enabled. An environment map generation procedure was developed and allows virtual objects to exhibit coherent effects such as color bleeding and specular reflection, even when the real objects are moved. The range of materials that can be rendered was widened by extending Lafortunes Spatial BRDF. The implemented infrastructure is offered as an authoring toolkit that consists of an API and a material editor tool. The aim of this authoring toolkit is to increase development productivity of Photorealistic Augmented Reality applications. The proposed solution was evaluated by taking into account visual and performance metrics. It allowed consistent rendering of dynamic scenes and photorealistic materials. The frame rate obtained was suitable to Augmented Reality applications when there were few virtual objects in the scene.

Texture-less planar object detection and pose estimation using Depth-Assisted Rectification of Contours

This paper presents a method named Depth-Assisted Rectification of Contours (DARC) for detection and pose estimation of texture-less planar objects using RGB-D cameras. It consists in matching contours extracted from the current image to previously acquired template contours. In order to achieve invariance to rotation, scale and perspective distortions, a rectified representation of the contours is obtained using the available depth information. DARC requires only a single RGB-D image of the planar objects in order to estimate their pose, opposed to some existing approaches that need to capture a number of views of the target object. It also does not require to generate warped versions of the templates, which is commonly needed by existing object detection techniques. It is shown that the DARC method runs in real-time and its detection and pose estimation quality are suitable for augmented reality applications.

[POSTER] Abecedary Tracking and Mapping: A Toolkit for Tracking Competitions

This paper introduces a toolkit with camera calibration, monocular visual Simultaneous Localization and Mapping (vSLAM) and registration with a calibration marker. With the toolkit, users can perform the whole procedure of the ISMAR on-site tracking competition in 2015. Since the source code is designed to be well-structured and highly-readable, users can easily install and modify the toolkit. By providing the toolkit, we encourage beginners to learn tracking techniques and to participate in the competition.

Evaluation of Motion Tracking and Depth Sensing Accuracy of the Tango Tablet

This paper presents an evaluation of the Tango tablet regarding its motion tracking and depth perception capabilities. A methodology for performing such kind of evaluation is proposed. Motion tracking error is assessed in both small workspace and large environments. In the small workspace scenario, the distances reported by the motion tracking system are compared with values measured with the aid of a graph paper. In the large environment condition, the tracking error consists in the difference between initial and final positions given by the system when the device moves around the environment and returns to the same location. Depth sensing precision is evaluated by comparing the 3D coordinates reported by the system of the inner corners of a chessboard pattern with ground truth values obtained from color camera image processing. The results show that Tango tablet sometimes presents large motion tracking errors, which may harm AR experience. In addition, Tango tablet depth sensing presents average error values similar to desktop depth cameras, but it is more sensitive to infrared reflection properties of the objects to be mapped.

Standalone edge-based markerless tracking of fully 3-dimensional objects for handheld augmented reality

This paper presents a markerless tracking technique targeted to the Windows Mobile Pocket PC platform. The primary aim of this work is to allow the development of standalone augmented reality applications for handheld devices based on natural feature tracking of fully 3-Dimensional objects. In order to achieve this goal, a model-based tracking approach that relies on edge information was adopted. Since it does not require high processing power, it is suitable for constrained devices such as handhelds. The OpenGL ES graphics library was used to detect the visible edges in a given frame, taking advantage of graphics hardware acceleration when available. In addition, a subset of two computer vision libraries was ported to the Pocket PC platform in order to provide some required algorithms to the markerless mobile solution. They were also adapted to use fixed-point math, with the purpose of improving the overall performance of the routines. The port of these libraries opens up the possibility of having other computer-vision tasks being executed on mobile platforms. An augmented reality application was created using the implemented technique and evaluations were done regarding tracking performance, accuracy and robustness. In most of the tests, the frame rates obtained are suitable for handheld augmented reality and a reasonable estimation of the object pose was provided.

Depth-assisted rectification for real-time object detection and pose estimation

RGB-D sensors have become in recent years a product of easy access to general users. They provide both a color image and a depth image of the scene and, besides being used for object modeling, they can also offer important cues for object detection and tracking in real time. In this context, the work presented in this paper investigates the use of consumer RGB-D sensors for object detection and pose estimation from natural features. Two methods based on depth-assisted rectification are proposed, which transform features extracted from the color image to a canonical view using depth data in order to obtain a representation invariant to rotation, scale and perspective distortions. While one method is suitable for textured objects, either planar or non-planar, the other method focuses on texture-less planar objects. Qualitative and quantitative evaluations of the proposed methods are performed, showing that they can obtain better results than some existing methods for object detection and pose estimation, especially when dealing with oblique poses.

ARBlocks: A Concept for a Dynamic Blocks Platform for Educational Activities

This paper describes the concept of a dynamic blocks platform, called ARBlocks, based on projective augmented reality and tangible user interfaces aiming educational activities. In it, the information is displayed by projectors, that exhibit the content only on the blocks using a projector calibration technique and the blocks are tracked through a frame marker. Despite the platform is still under development, some results regarding the frame marker tracking and the projection was achieved.

RPR-SORS: An Authoring Toolkit for Photorealistic AR

This paper presents a solution for photorealistic rendering of virtual objects into dynamic real scenes. It consists of a toolkit for Augmented Reality, named RPR- SORS, which is composed of two artifacts: an API that handles various computer graphics techniques in a novel pipeline approaching illumination, reflectance model, shadowing, composition, and camera effects; and a material editor that accelerates the creation of complex photorealistic materials. An experimental application was developed with the proposed solution, where both the API and material editor were used to accomplish photorealistic effects in an application of Augmented Reality in architecture. The results obtained show that the API provides a flexible infrastructure for the photorealistic Augmented Reality pipeline, and that the material editor facilitates the creation of photorealistic materials and their use for Augmented Reality

FPGA infrastructure for the development of augmented reality applications

In this paper, we describe a platform that aims to help developers on the construction of embedded AR applications. The infrastructure is based on Field Programmable Gate Array and enables the creation of hardware based AR systems, contributing to high speed processing and low power consumption. In addition, the platform uses a componentized design model, where the steps needed to develop an AR application consist in combining the necessary components. A number of image processing algorithms have been implemented and some AR applications have been created using this infrastructure in order to demonstrate the platform capabilities. The performance obtained with the hardware version (at 100 MHz) of the algorithms has also been compared with their software counterparts (at 2 GHz).