Vitor Guizilini

Visual odometry learning for unmanned aerial vehicles

This paper addresses the problem of using visual information to estimate vehicle motion (a.k.a. visual odometry) from a machine learning perspective. The vast majority of current visual odometry algorithms are heavily based on geometry, using a calibrated camera model to recover relative translation (up to scale) and rotation by tracking image features over time. Our method eliminates the need for a parametric model by jointly learning how image structure and vehicle dynamics affect camera motion. This is achieved with a Gaussian Process extension, called Coupled GP, which is trained in a supervised manner to infer the underlying function mapping optical flow to relative translation and rotation. Matched image features parameters are used as inputs and linear and angular velocities are the outputs in our non-linear multi-task regression problem. We show here that it is possible, using a single uncalibrated camera and establishing a first-order temporal dependency between frames, to jointly estimate not only a full 6 DoF motion (along with a full covariance matrix) but also relative scale, a non-trivial problem in monocular configurations. Experiments were performed with imagery collected with an unmanned aerial vehicle (UAV) flying over a deserted area at speeds of 100–120 km/h and altitudes of 80-100 m, a scenario that constitutes a challenge for traditional visual odometry estimators.

Online Self-Supervised Multi-Instance Segmentation of Dynamic Objects

This paper presents a method for the continuous segmentation of dynamic objects using only a vehicle mounted monocular camera without any prior knowledge of the objects appearance. Prior work in online static/dynamic segmentation [1] is extended to identify multiple instances of dynamic objects by introducing an unsupervised motion clustering step. These clusters are then used to update a multi-class classifier within a self-supervised framework. In contrast to many tracking-by-detection based methods, our system is able to detect dynamic objects without any prior knowledge of their visual appearance shape or location. Furthermore, the classifier is used to propagate labels of the same object in previous frames, which facilitates the continuous tracking of individual objects based on motion. The proposed system is evaluated using recall and false alarm metrics in addition to a new multi-instance labelled dataset to measure the performance of segmenting multiple instances of objects.

Semi-parametric learning for visual odometry

This paper addresses the visual odometry problem from a machine learning perspective. Optical flow information from a single camera is used as input for a multiple-output Gaussian process (MOGP) framework, that estimates linear and angular camera velocities. This approach has several benefits. (1) It substitutes the need for conventional camera calibration, by introducing a semi-parametric model that is able to capture nuances that a strictly parametric geometric model struggles with. (2) It is able to recover absolute scale if a range sensor (e.g. a laser scanner) is used for ground-truth, provided that training and testing data share a certain similarity. (3) It is naturally able to provide measurement uncertainties. We extend the standard MOGP framework to include the ability to infer joint estimates (full covariance matrices) for both translation and rotation, taking advantage of the fact that all estimates are correlated since they are derived from the same vehicle. We also modify the common zero mean assumption of a Gaussian process to accommodate a standard geometric model of the camera, thus providing an initial estimate that is then further refined by the non-parametric model. Both Gaussian process hyperparameters and camera parameters are trained simultaneously, so there is still no need for traditional camera calibration, although if these values are known they can be used to speed up training. This approach has been tested in a wide variety of situations, both 2D in urban and off-road environments (two degrees of freedom) and 3D with unmanned aerial vehicles (six degrees of freedom), with results that are comparable to standard state-of-the-art visual odometry algorithms and even more traditional methods, such as wheel encoders and laser-based Iterative Closest Point. We also test its limits to generalize over environment changes by varying training and testing conditions independently, and also by changing cameras between training and testing.

Semi-parametric models for visual odometry

This paper introduces a novel framework for estimating the motion of a robotic car from image information, a scenario widely known as visual odometry. Most current monocular visual odometry algorithms rely on a calibrated camera model and recover relative rotation and translation by tracking image features and applying geometrical constraints. This approach has some drawbacks: translation is recovered up to a scale, it requires camera calibration which can be tricky under certain conditions, and uncertainty estimates are not directly obtained. We propose an alternative approach that involves the use of semi-parametric statistical models as means to recover scale, infer camera parameters and provide uncertainty estimates given a training dataset. As opposed to conventional non-parametric machine learning procedures, where standard models for egomotion would be neglected, we present a novel framework in which the existing parametric models and powerful non-parametric Bayesian learning procedures are combined. We devise a multiple output Gaussian Process (GP) procedure, named Coupled GP, that uses a parametric model as the mean function and a non-stationary covariance function to map image features directly into vehicle motion. Additionally, this procedure is also able to infer joint uncertainty estimates (full covariance matrices) for rotation and translation. Experiments performed using data collected from a single camera under challenging conditions show that this technique outperforms traditional methods in trajectories of several kilometers.

Automatic detection of Ceratocystis wilt in Eucalyptus crops from aerial images

One of the challenges in precision agriculture is the detection of diseased crops in agricultural environments. This paper presents a methodology to detect the Ceratocystis wilt disease in Eucalyptus crops. An unmanned aerial vehicle is used to obtain high-resolution RGB images of a predefined area. The methodology enables the extraction of visual features from image regions and uses several supervised machine learning (ML) techniques to classify regions into three classes: ground, healthy and diseased plants. Several learning techniques were compared using data obtained from a commercial Eucalyptus plantation. Experimental results show that the GP learning model is more reliable than the other learning methods for accurately identifying diseased trees.

Online self-supervised segmentation of dynamic objects

We address the problem of automatically segmenting dynamic objects in an urban environment from a moving camera without manual labelling, in an online, self-supervised learning manner. We use input images obtained from a single uncalibrated camera placed on top of a moving vehicle, extracting and matching pairs of sparse features that represent the optical flow information between frames. This optical flow information is initially divided into two classes, static or dynamic, where the static class represents features that comply to the constraints provided by the camera motion and the dynamic class represents the ones that do not. This initial classification is used to incrementally train a Gaussian Process (GP) classifier to segment dynamic objects in new images. The hyperparameters of the GP covariance function are optimized online during navigation, and the available self-supervised dataset is updated as new relevant data is added and redundant data is removed, resulting in a near-constant computing time even after long periods of navigation. The output is a vector containing the probability that each pixel in the image belongs to either the static or dynamic class (ranging from 0 to 1), along with the corresponding uncertainty estimate of the classification. Experiments conducted in an urban environment, with cars and pedestrians as dynamic objects and no prior knowledge or additional sensors, show promising results even when the vehicle is moving at considerable speeds (up to 50 km/h). This scenario produces a large quantity of featureless regions and false matches that is very challenging for conventional approaches. Results obtained using a portable camera device also testify to our algorithms ability to generalize over different environments and configurations without any fine-tuning of parameters.

Learning to Reconstruct 3D Structures for Occupancy Mapping

Real world scenarios contain many structural patterns that, if appropriately extracted and modeled, can be used to reduce problems associated with sensor failure and occlusions, while improving planning methods in tasks such as navigation and grasping. This paper devises a novel unsupervised procedure that is able to learn 3D structures from unorganized point clouds as occupancy maps. Our framework enables the learning of unique and arbitrarily complex features using a Bayesian Convolutional Variational Auto-Encoder that compresses local information into a latent low-dimensional representation and then decodes it back in order to reconstruct the original scene. This reconstructive model is trained on features obtained automatically from a wide variety of scenarios to improve its generalization and interpolative powers. We show that the proposed framework is able to recover partially missing structures and reason over occlusion with high accuracy, while maintaining a detailed reconstruction of observed areas. To seamlessly combine this localized feature information into a single global structure, we employ a Hilbert Map, recently proposed as a robust and efficient occupancy mapping technique. Experimental tests are conducted in large-scale 2D and 3D datasets, and a study on the impact of various accuracy/speed trade-offs is provided to assess the limits of the proposed framework.

The Impact of DoS Attacks on the AR.Drone 2.0

A key challenge for the use of unmanned aerial vehicles (UAVs) is the security of their information during navigation to accomplish its task. Information security is a known issue, but it seems to be overlooked from a research perspective, that tends to focus on more classical and well-formulated problems. This paper addresses an experimental evaluation of three Denial of Service (DoS) attack tools to analyze the UAVs behavior. These tools are executed in real-time on the robot while it navigates an indoor environment (inside the University building). We present experiments to demonstrate the impact of such attacks on a particular UAV model (AR.Drone 2.0) and also show a description of existing vulnerabilities. Our results indicate that DoS attacks might cause network availability issues influencing critical UAVs applications, such as the video streaming functionality.

Route planning for active classification with UAVs

The mapping of agricultural crops by capturing images obtained with UAVs enables fast environmental monitoring and diagnosis in large areas. Airborne monitoring in agriculture can a substantially impacts on the identification of diseases and produce accurate information on affected areas. The problem can be formulated as a classification task on aerial images with significant opportunities to impact other fields. This paper presents an active learning method through route planning for improvements in the knowledge on visited areas and minimization uncertainties about the classification of diseases in crops. Binary Logistic Regression and Gaussian Process were used for the detection of pathologies and map interpolation, respectively. A Bayesian optimization strategy is also proposed for the planning of an informative trajectory, which resulted in a maximized search for affected areas in an initially unknown environment.

Multi-task Learning of Visual Odometry Estimators

This paper presents a novel framework for learning visual odometry estimators from a single uncalibrated camera through multi-task non-parametric Bayesian inference. A new methodology, Coupled Gaussian Processes, is developed to jointly estimate vehicle velocity while concomitantly inferring a full covariance matrix of all tasks. Matched image feature descriptors obtained from sequential frames act as inputs and the vehicle’s linear and angular velocities as outputs, allowing its position to be incrementally determined. This approach has three main benefits: firstly, it readily provides uncertainty measurements, thus allowing posterior data fusion with other sensors; secondly, it eliminates the need for camera calibration, as the system essentially learns the transformation between the optical flow and vehicle velocity spaces; thirdly, it provides motion estimation directly, not subject to scaling as in standard structure from motion techniques with monocular cameras. Experiments conducted using imagery collected in urban and off-road environments under challenging conditions show the benefits of the approach for trajectories of up to 2 km. Finally, the framework is integrated into a Exactly Sparse Extended Information Filter for deployment in a SLAM scenario.