Geraldo Silveira

Real-time Visual Tracking under Arbitrary Illumination Changes

In this paper, we investigate how to improve the robustness of visual tracking methods with respect to generic lighting changes. We propose a new approach to the direct image alignment of either Lambertian or non-Lambertian objects under shadows, inter-reflections, glints as well as ambient, diffuse and specular reflections which may vary in power, type, number and space. The method is based on a proposed model of illumination changes together with an appropriate geometric model of image motion. The parameters related to these models are obtained through an efficient second-order optimization technique which minimizes directly the intensity discrepancies. Comparison results with existing direct methods show significant improvements in the tracking performance. Extensive experiments confirm the robustness and reliability of our method.

Unified Direct Visual Tracking of Rigid and Deformable Surfaces Under Generic Illumination Changes in Grayscale and Color Images

The fundamental task of visual tracking is considered in this work as an incremental direct image registration problem. Direct methods refer to those that exploit the pixel intensities without resorting to image features. We propose new transformation models and optimization methods for directly and robustly registering images (including color ones) of rigid and deformable objects, all in a unified manner. We also show that widely adopted models are in fact particular cases of the proposed ones. Indeed, the proposed general models combine various classes of image warps and ensure robustness to generic lighting changes. Finally, the proposed optimization method together with the exploitation of all possible image information allow the algorithm to achieve high levels of accuracy. Extensive experiments are reported to demonstrate that visual tracking can indeed be highly accurate and robust despite deforming objects and severe illumination changes.

A nonlinear observer approach for concurrent estimation of pose, IMU bias and camera-to-IMU rotation

This paper concerns the problem of pose estimation for an inertial-visual sensor. It is well known that IMU bias, and calibration errors between camera and IMU frames can impair the achievement of high-quality estimates through the fusion of visual and inertial data. The main contribution of this work is the design of new observers to estimate pose, IMU bias and camera-to-IMU rotation. The observers design relies on an extension of the so-called passive complementary filter on SO(3). Stability of the observers is established using Lyapunov functions under adequate observability conditions. Experimental results are presented to assess this approach.

A fast vision-based road following strategy applied to the control of aerial robots

The utilization of the vision as a feedback sensor in closed-loop control schemes is of great interest, mainly due to the high density of information revealed by images. In this work, we present a method to perform road following tracking by aerial unmanned vehicles (AUV) based on visual input. Difficulties arise from the nonholonomic constraints of the AUV moving in 3D. The problems are overcome using a visual servoing approach based on an interaction matrix. Simulation results validating the strategy are shown.

Photogeometric Direct Visual Tracking for Central Omnidirectional Cameras

The fundamental task of visual tracking is considered in this work as a direct image registration problem. Direct methods refer to those that exploit the pixel intensities without intermediate steps, e.g., no extraction of image features. This article proposes new photogeometric transformation models and nonlinear optimization methods for directly registering central omnidirectional images. The proposed models ensure robustness to arbitrary illumination changes, as well as encompass all classes of projective deformations of planar objects within those images. Experimental results show that visual tracking can indeed be highly robust and accurate even for this type of vision systems.

Visual servoing from robust direct color image registration

To date, there exist only few works on the use of color images for visual servoing. Perhaps, this is due to the difficulties usually found to cope with illumination changes in these images. This paper presents new parametric models and optimization methods for robustly and directly registering color images. Direct methods refer to those that exploit the pixel intensities, without resorting to image features. We then show how a robust and generic visual servoing scheme can be constructed using the obtained optimal parameters. The proposed models ensure robustness to arbitrary illumination changes in color images, do not require prior knowledge (including the spectral ones) of the object, illuminants or camera, and naturally encompass gray-level images. Furthermore, the exploitation of all information within the images, even from areas where no features exist, allow the algorithm to achieve high levels of accuracy. Various results are reported to show that visual servoing can indeed be highly accurate and robust despite unknown objects and unknown imaging conditions.

On intensity-based 3D visual servoing

Abstract This article investigates the problem of pose-based visual servoing whose equilibrium state is defined via a reference image. Differently from most solutions, this work directly exploits the pixel intensities without any feature extraction or matching. Intensity-based methods provide for higher accuracy and versatility. Another central idea of this work concerns the exploitation of the observability issue associated to monocular systems, which always occurs around the equilibrium. This overall framework allows for developing a family of new 3D visual servoing techniques with varying degrees of computational complexity and of prior knowledge. Importantly, they are arranged hierarchically from the simplest to the optimal ones, all in a unified scheme. Three new methods are then presented, and their closed-loop performances are experimentally assessed using a six degree-of-freedom robotic arm. As an additional contribution, these results refute the common belief that correct camera calibration and pose recovery are crucial to the accuracy of 3D visual servoing techniques.

Decoupled direct visual servoing

This article addresses the problem of direct vision-based robot control where the equilibrium state is defined via a reference image. Direct methods refer to intensity-based nonmetric techniques to perform that stabilization. Intensity-based strategies provide for higher accuracy, whereas not requiring any metric information improves their versatility. However, existing direct techniques either have a coupled error dynamics, or are designed for planar objects only. This paper proposes a new direct technique that decouples the translational motion from the rotational one for the general case of both planar and nonplanar targets under general translational and rotational displacements. Furthermore, for the important case of a fronto-parallel planar object, the proposed technique leads to a fully diagonal interaction matrix. The equilibrium state is made locally exponentially stable for all those cases. These improvements are theoretically proven and experimentally demonstrated using a 6-DoF robotic arm.

Reference image computation from cartesian specifications for model-free visual servoing

Abstract It is proposed a new method to compute, from the current unstructured view, a reference (desired) image for visual servoing. The approach, which is based on Cartesian specifications, is made without moving the robot to the target location as in the teach-by-showing paradigm. In addition, although it is assumed a piecewise planar target, no metric information between the object features is required. Several results demonstrate its validity and usefulness, for example in reconnaissance and surveillance tasks.