Silvio Ricardo Rodrigues Sanches

Mutual occlusion between real and virtual elements in Augmented Reality based on fiducial markers

Augmented Reality (AR) systems which use optical tracking with fiducial marker for registration have had an important role in popularizing this technology, since only a personal computer with a conventional webcam is required. However, in most these applications, the virtual elements are shown only in the foreground a real element does not occlude a virtual one. The method presented enables AR environments based on fiducial markers to support mutual occlusion between a real element and many virtual ones, according to the elements position (depth) in the environment.

Subjective Video Quality Assessment in Segmentation for Augmented Reality Applications

Video segmentation to extract a person in foreground has been a common task in many Augmented Reality (AR) applications. In natural environments which the background color and the light environment are not constant the segmentation method must be able to extract the element of the interest in these conditions. However, methods for segmentation in natural environment are more error prone than the traditional ones which are based on a constant color elimination. Thus, in order to use them in a large number of applications it is necessary to know how the segmentation errors are perceived by the users in order to focus on development of algorithms which avoid the more perceptible ones. In this work a video quality subjective assessment method was applied to obtain the AR users opinions about videos with different misclassified pixels rates. The results showed that segmentation errors are perceived by AR applications users. However, the video quality was not related with the number of the misclassified pixels. In addiction, it was noted that when the errors concentrated in the element of the interest increase the score of the associated video decreases.

The Generation of Scenes in Mixed Reality Environments using the Chromakey Technique

A classic problem in the development of mixed reality systems is the registration. The correct alignment between virtual objects and the real elements is extremely important for the coherent composition of the resultant scene. Considering this context, this paper describes an approach for the composition of scenes in mixed reality environments using the chromakey technique for the extraction of real objects. After that, the scene is mounted in a coherent way related to the depth in OpenGL framebuffer for posterior rendering.

Bilayer segmentation augmented with future evidence

This paper presents an algorithm that augments a previous model known in the literature for the automatic segmentation of monocular videos into foreground and background layers. The original model fuses visual cues such as color, contrast, motion and spatial priors within a Conditional Random Field. Our augmented model makes use of bidirectional motion priors by exploiting future evidence. Although our augmented model processes more data, it does so with the same time performance of the original model. We evaluate the augmented model within ground truth data and the results show that the augmented model produces better segmentation.

Immersive Teleconferencing System Based on Video-Avatar for Distance Learning

Teleconference is an important means of communication in distance learning courses. Unlike other technologies used in education, the technology that enables virtual conferencing has changed little since it began to be used. Even the transition from analog to digital technology and the adoption of IP technology changed little in the communication characteristics of this media, basically the same of 2D. The project presented here is part of a new field of research, called immersive telecommunication, which aims at increasing the sense of immersion and presence of the participants. For this purpose, a videoconferencing system was developed, which allows presenters to immerse themselves in a 3D virtual environment, through their video-avatar, and be watched remotely in the form of 3D video. This system integrates several subprojects, involving the capture and rendering of a video-avatar, background removal, gesture recognition, interaction design, among others. The aim of this paper is to present the system architecture and an overview of its modules, the key challenges and adopted solutions.

Objective assessment of video segmentation quality for augmented reality

Assessment of video segmentation quality is a problem seldom investigated by the scientific community. Nevertheless, recent studies presented some objective metrics to evaluate algorithms. Such metrics consider different ways in which segmentation errors occur (perceptual factors) and its parameters are adjusted according to the application for which the segmented frames are intended. We demonstrate empirically that the performance of existing metrics changes according to the segmentation algorithm; we applied such metrics to evaluate bilayer segmentation algorithms for compose scenes in Augmented Reality environments. We also contribute with a new objective metric to adjust the parameters of two bilayer segmentation algorithms found in the literature.

Bilayer Segmentation of Live Video in Uncontrolled Environments for Background Substitution: An Overview and Main Challenges

Bilayer segmentation of live video in uncontrolled environments is an essential task for home applications in which the original background of the scene must be replaced, as in video chats or traditional videoconference. The main challenge in such conditions is overcome all difficulties in problem-situations (e.g., illumination change, distract events such as element moving in the background and camera shake) that may occur while the video is being captured. This paper presents a survey of segmentation methods for background substitution applications, describes the main concepts and identifies events that may cause errors. Our analysis shows that although robust methods rely on specific devices (multiple cameras or sensors to generate depth maps) which aid the process. In order to achieve the same results using conventional devices (monocular video cameras), most current research relies on energy minimization frameworks, in which temporal and spacial information are probabilistically combined with those of color and contrast.

Aspects of User Profiles That Can Improve Mobile Augmented Reality Usage

Augmented Reality (AR) applications running on mobile devices have become more popular in recent years. As with digital games, many developers have begun to direct their applications to mobile platforms, although the processing power of these devices is usually smaller when compared to the computing power of laptops and desktops. Applications developed for computing devices, including mobile devices, typically have a well-defined target audience, and some of them use AR technology. The use of AR can actually add value to the application, but despite the additional motivation, interaction in this type of environment may be more complex compared to traditional applications. Considering this context, the main objective of the present work is to identify which key aspects in the user profile impact the design of mobile AR applications.

Segmentation Quality for Augmented Reality: An Objective Metric

Assessment the quality of segmentation algorithms considering the user perception is an important problem in Computer Vision. For this purpose a metric must take into account the impact of different types of errors displayed to the users. In this work we developed a new objective metric to assess the quality obtained by bilayer segmentation algorithms when they are used in Augmented Reality applications. Our metric proved to be efficient for selecting the best algorithm and to adjust its parameters.