Luis Ferraz

Discriminative Learning of Deep Convolutional Feature Point Descriptors

Deep learning has revolutionalized image-level tasks such as classification, but patch-level tasks, such as correspondence, still rely on hand-crafted features, e.g. SIFT. In this paper we use Convolutional Neural Networks (CNNs) to learn discriminant patch representations and in particular train a Siamese network with pairs of (non-)corresponding patches. We deal with the large number of potential pairs with the combination of a stochastic sampling of the training set and an aggressive mining strategy biased towards patches that are hard to classify. By using the L2 distance during both training and testing we develop 128-D descriptors whose euclidean distances reflect patch similarity, and which can be used as a drop-in replacement for any task involving SIFT. We demonstrate consistent performance gains over the state of the art, and generalize well against scaling and rotation, perspective transformation, non-rigid deformation, and illumination changes. Our descriptors are efficient to compute and amenable to modern GPUs, and are publicly available.

Very Fast Solution to the PnP Problem with Algebraic Outlier Rejection

We propose a real-time, robust to outliers and accurate solution to the Perspective-n-Point (PnP) problem. The main advantages of our solution are twofold: first, it in- tegrates the outlier rejection within the pose estimation pipeline with a negligible computational overhead, and sec- ond, its scalability to arbitrarily large number of correspon- dences. Given a set of 3D-to-2D matches, we formulate pose estimation problem as a low-rank homogeneous sys- tem where the solution lies on its 1D null space. Outlier correspondences are those rows of the linear system which perturb the null space and are progressively detected by projecting them on an iteratively estimated solution of the null space. Since our outlier removal process is based on an algebraic criterion which does not require computing the full-pose and reprojecting back all 3D points on the image plane at each step, we achieve speed gains of more than 100× compared to RANSAC strategies. An extensive exper- imental evaluation will show that our solution yields accu- rate results in situations with up to 50% of outliers, and can process more than 1000 correspondences in less than 5ms.

Concealed object detection and segmentation over millimetric waves images

Millimetric Waves Images (MMW) are becoming more and more useful in the passive detection of threaten objects based on plastic substances as explosives or sharp/cutting weapons. Our goal is to achieve segmentation of the body and concealed threats dealing with the inherent problems of this type of images: noise, low resolution and intensity inhomogeneity. In this work we present the results of applying Iterative Steering Kernel Regression (ISKR) method for denoising and Local Binary Fitting (LBF) for segmentation in order to correctly segment bodies and threats over a database of 29 MMW images. These methods, which had not been tested in the literature with these type of images, are compared with previously applied state of the art methods. Experimental results show that the use of the proposed methods in MMW images improve the results that had been obtained before.

Multiple Kernel Two-Step Tracking

In tracking tasks, representing a target region as a weighted histogram has opened possibilities which led to excellent results, as mean shift or camshift algorithms. This representation is extracted from the image by giving weights with kernels and it depends on the properties of the kernels. By a first order Taylor approximation of the histograms it is possible to perform a tracking using several kernels, interpreted as different sources of information. This representation improves the possibilities and gives more flexibility when facing problems of tracking, as occlusions, model variance or projective deformations of the image. In this paper we use this multi-kernel model representation to perform a simultaneous tracking of the entire object and also of each different part individually. This is performed in a new two-step process. In the first step we perform the multikernel estimation and in a second step we update the model representation taking into account single kernel estimations, representing the local movement of each part. From a probabilistic view of the Matusita metric we analyze the usefulness of this method against partial occlusions and some projective transformations like zooms or 3D rotations and articulated movements.

A sparse curvature-based detector of affine invariant blobs

Usually, state-of-the-art interest point detectors tend to over-represent the local image structures associating several interest points for each local image structure. This fact avoids interest points to clearly stand out against its neighborhood, losing the ability to clearly describe the global uniqueness of each local image structure. In order to solve this problem we propose a sparse affine invariant blob detector, which tries to describe each blob structure with a single interest point. The proposed detector is carried out in two stages: an initial stage, where a set of scale invariant interest points are located by means of the idea of blob movement and blob evolution (creation, annihilation and merging) along different scales by using a precise description of the image provided by the Gaussian curvature, providing a global bottom-up estimation of the image structure. During the second stage, the shape and location of each scale invariant interest point is refined by fitting an anisotropic Gaussian function, which minimizes the error with the underlying image and simultaneously estimates both the shape and location, by means of a non-linear least squares approach. A comparative evaluation of affine invariant detectors is presented, showing that our approach outperforms state-of-the-art affine invariant detectors in terms of precision and recall, and obtains a similar performance to that of the best ones in terms of repeatability and matching. In addition we demonstrate that our detector does not over-represent blob structures and provides a sparse detection that improves distinctiveness and reduces drastically the computational cost of matching tasks. In order to verify the accuracy and the reduction in the computational cost we have evaluated our detector in image registration tasks.

Leveraging feature uncertainty in the PnP problem

We propose a real-time and accurate solution to the Perspective-n-Point (PnP) problem –estimating the pose of a calibrated camera from n 3D-to-2D point correspondences– that exploits the fact that in practice the 2D position of not all 2D features is estimated with the same accuracy. Assuming a model of such feature uncertainties is known in advance, we reformulate the PnP problem as a maximum likelihood minimization approximated by an unconstrained Sampson error function, which naturally penalizes the most noisy correspondences. The advantages of this approach are thoroughly demonstrated in synthetic experiments where feature uncertainties are exactly known. Pre-estimating the features uncertainties in real experiments is, though, not easy. In this paper we model feature uncertainty as 2D Gaussian distributions representing the sensitivity of the 2D feature detectors to different camera viewpoints. When using these noise models with our PnP formulation we still obtain promising pose estimation results that outperform the most recent approaches.

Efficient monocular pose estimation for complex 3D models

We propose a robust and efficient method to estimate the pose of a camera with respect to complex 3D textured models of the environment that can potentially contain more than 100; 000 points. To tackle this problem we follow a top down approach where we combine high-level deep network classifiers with low level geometric approaches to come up with a solution that is fast, robust and accurate. Given an input image, we initially use a pre-trained deep network to compute a rough estimation of the camera pose. This initial estimate constrains the number of 3D model points that can be seen from the camera viewpoint. We then establish 3D-to-2D correspondences between these potentially visible points of the model and the 2D detected image features. Accurate pose estimation is finally obtained from the 2D-to-3D correspondences using a novel PnP algorithm that rejects outliers without the need to use a RANSAC strategy, and which is between 10 and 100 times faster than other methods that use it. Two real experiments dealing with very large and complex 3D models demonstrate the effectiveness of the approach.

Optimization of the SSD multiple kernel tracking applied to IR video sequences

This paper addresses the problem of tracking a target in an IR video sequence using a kernel based histogram representation of the target. In this field, gradient ascent methods have demonstrated useful results with weighted kernels and in particular Mean Shift is currently the most commonly used gradient scale method. Our approximation follows the work made by Hager, that uses a SSD objective function (derived from Matusita metric) and combines it with a Newton-like maximization method, resulting a fast gradient scale system. An important property is that this method enables the use of multiple kernels, allowing a more powerful representation with a minimum increasing of computational cost. We analyse the limitation of this representation using the Newton maximization algorithm and we introduce the concept of direction of ambiguity. This concept allows a criterion for choosing the kernels that drive the iteration to minimize the error criterion. The results we present show the improvements of the method over a tracking problem. The target is a small car with a great background similarity.