Elif Vural

Analysis of Descent-Based Image Registration

We present a performance analysis for image registration with gradient descent. We consider a typical multiscale registration setting where the global two-dimensional translation between a pair of images is estimated by smoothing the images and minimizing the distance between them with gradient descent. Our study particularly concentrates on the effect of noise and low-pass filtering on the alignment accuracy. We analyze the well-behavedness of the image distance function by estimating the neighborhood of translations for which it is free of undesired local minima. This is the neighborhood of translations that are correctly computable with a simple gradient descent minimization. We show that the area of this neighborhood increases at least quadratically with the smoothing filter size. We then examine the effect of noise on the alignment accuracy and derive an upper bound for the alignment error in terms of the noise properties and filter size. Our main finding is that the error increases at a rate that is...

Out-of-Sample Generalizations for Supervised Manifold Learning for Classification

Supervised manifold learning methods for data classification map high-dimensional data samples to a lower dimensional domain in a structure-preserving way while increasing the separation between different classes. Most manifold learning methods compute the embedding only of the initially available data; however, the generalization of the embedding to novel points, i.e., the out-of-sample extension problem, becomes especially important in classification applications. In this paper, we propose a semi-supervised method for building an interpolation function that provides an out-of-sample extension for general supervised manifold learning algorithms studied in the context of classification. The proposed algorithm computes a radial basis function interpolator that minimizes an objective function consisting of the total embedding error of unlabeled test samples, defined as their distance to the embeddings of the manifolds of their own class, as well as a regularization term that controls the smoothness of the interpolation function in a direction-dependent way. The class labels of test data and the interpolation function parameters are estimated jointly with an iterative process. Experimental results on face and object images demonstrate the potential of the proposed out-of-sample extension algorithm for the classification of manifold-modeled data sets.

Learning Smooth Pattern Transformation Manifolds

Manifold models provide low-dimensional representations that are useful for processing and analyzing data in a transformation-invariant way. In this paper, we study the problem of learning smooth pattern transformation manifolds from image sets that represent observations of geometrically transformed signals. To construct a manifold, we build a representative pattern whose transformations accurately fit various input images. We examine two objectives of the manifold-building problem, namely, approximation and classification. For the approximation problem, we propose a greedy method that constructs a representative pattern by selecting analytic atoms from a continuous dictionary manifold. We present a dc optimization scheme that is applicable to a wide range of transformation and dictionary models, and demonstrate its application to the transformation manifolds generated by the rotation, translation, and anisotropic scaling of a reference pattern. Then, we generalize this approach to a setting with multiple transformation manifolds, where each manifold represents a different class of signals. We present an iterative multiple-manifold-building algorithm such that the classification accuracy is promoted in the learning of the representative patterns. The experimental results suggest that the proposed methods yield high accuracy in the approximation and classification of data compared with some reference methods, while the invariance to geometric transformations is achieved because of the transformation manifold model.

Geometry-Aware Neighborhood Search for Learning Local Models for Image Superresolution

Local learning of sparse image models has proved to be very effective to solve inverse problems in many computer vision applications. To learn such models, the data samples are often clustered using the K-means algorithm with the Euclidean distance as a dissimilarity metric. However, the Euclidean distance may not always be a good dissimilarity measure for comparing data samples lying on a manifold. In this paper, we propose two algorithms for determining a local subset of training samples from which a good local model can be computed for reconstructing a given input test sample, where we consider the underlying geometry of the data. The first algorithm, called adaptive geometry-driven nearest neighbor search (AGNN), is an adaptive scheme, which can be seen as an out-of-sample extension of the replicator graph clustering method for local model learning. The second method, called geometry-driven overlapping clusters (GOCs), is a less complex nonadaptive alternative for training subset selection. The proposed AGNN and GOC methods are evaluated in image superresolution and shown to outperform spectral clustering, soft clustering, and geodesic distance-based subset selection in most settings.

Discretization of Parametrizable Signal Manifolds

Transformation-invariant analysis of signals often requires the computation of the distance from a test pattern to a transformation manifold. In particular, the estimation of the distances between a transformed query signal and several transformation manifolds representing different classes provides essential information for the classification of the signal. In many applications, the computation of the exact distance to the manifold is costly, whereas an efficient practical solution is the approximation of the manifold distance with the aid of a manifold grid. In this paper, we consider a setting with transformation manifolds of known parameterization. We first present an algorithm for the selection of samples from a single manifold that permits to minimize the average error in the manifold distance estimation. Then we propose a method for the joint discretization of multiple manifolds that represent different signal classes, where we optimize the transformation-invariant classification accuracy yielded by the discrete manifold representation. Experimental results show that sampling each manifold individually by minimizing the manifold distance estimation error outperforms baseline sampling solutions with respect to registration and classification accuracy. Performing an additional joint optimization on all samples improves the classification performance further. Moreover, given a fixed total number of samples to be selected from all manifolds, an asymmetric distribution of samples to different manifolds depending on their geometric structures may also increase the classification accuracy in comparison with the equal distribution of samples.

Outlier Removal for Sparse 3D Reconstruction from Video

In this work, a comparative study on sparse metric 3D reconstruction from typical video content is presented. Experimental tests are performed in order to evaluate the performances of competing algorithms from the literature in several stages of 3D reconstruction, such as feature detection and epipolar geometry estimation. During the simulations, competing algorithms, such as SIFT and Harris corner detector, PROSAC and RANSAC, 7-point and 8-point algorithms, are tested on various video content, such as TV broadcasts or recording by a hand-held camera in a controlled environment. Based on these results, it could be concluded that SIFT yields significant improvements over Harris in terms of the quality of correspondences between frames, whereas RANSAC and PROSAC perform similarly for the case of limited outliers. Finally, 7-point algorithm yields slightly superior results over 8-point. In addition to this comparative study, a novel method for the elimination of erroneous points from the reconstructed scene is proposed. The quality of the resulting algorithm is quite acceptable for its 3D reconstruction.

Approximation of pattern transformation manifolds with parametric dictionaries

The construction of low-dimensional models explaining high-dimensional signal observations provides concise and efficient data representations. In this paper, we focus on pattern transformation manifold models generated by in-plane geometric transformations of 2D visual patterns. We propose a method for computing a manifold by building a representative pattern such that its transformation manifold accurately fits a set of given observations. We present a solution for the progressive construction of the representative pattern with the aid of a parametric dictionary, which in turn provides an analytical representation of the data and the manifold. Experimental results show that the patterns learned with the proposed algorithm can efficiently capture the main characteristics of the input data with high approximation accuracy, where the invariance to the geometric transformations of the data is accomplished due to the transformation manifold model.

Distance-based discretization of parametric signal manifolds

The characterization of signals and images in manifolds often lead to efficient dimensionality reduction algorithms based on manifold distance computation for analysis or classification tasks. We propose in this paper a method for the discretization of signal manifolds given in a parametric form. We present an iterative algorithm for the selection of samples on the manifold that permits to minimize the average error in the manifold distance computation. Experimental results with image appearance manifolds demonstrate that the proposed discretization algorithm outperforms baseline solutions based on random or regular sampling, both in terms of projection accuracy and image registration.

Domain adaptation via transferring spectral properties of label functions on graphs

We propose a domain adaptation algorithm that relies on a graph representation of data samples in the source and target domains. The algorithm combines the information of the known class labels in the source and target domains through the Fourier coefficients of the class label function in the two graphs. The proposed method does not require an ordering or a one-to-one mapping between the samples of the source and target domains; instead, it uses only a small set of matched pairs that serve the purpose of “aligning” the source and target Fourier bases. The estimation of the coefficients of the label function in the source and target Fourier bases is then formulated as a simple convex optimization problem. The proposed domain adaptation algorithm is tested in face recognition under varying pose and illumination and is observed to provide significant improvement over reference classification approaches especially when the data distributions in the source and target domains differ significantly.

Analysis of image registration with tangent distance

The computation of the geometric transformation between a reference and a target image, known as registration or alignment, corresponds to the projection of the target image onto the transformation manifold of the reference image (the set of images generated by its geometric transformations). However, it often takes a nontrivial form such that the exact computation of projections on the manifold is difficult. The tangent distance method is an effective algorithm for solving this problem by exploiting a linear approximation of the transformation manifold of the reference image. As theoretical studies about the tangent distance algorithm have been largely overlooked, we present in this work a detailed performance analysis of this useful algorithm, which can eventually help its implementation and the selection of its parameters. We consider a popular image registration setting using a multiscale pyramid of low-pass filtered versions of the (possibly noisy) reference and target images, which is particularly u...