Dennis Strelow

Motion estimation from image and inertial measurements

Cameras and inertial sensors are each good candidates for autonomous vehicle navigation, modeling from video, and other applications that require six-degrees-of-freedom motion estimation. However, these sensors are also good candidates to be deployed together, since each can be used to resolve the ambiguities in estimated motion that result from using the other modality alone. In this paper, we consider the specific problem of estimating sensor motion and other unknowns from image, gyro, and accelerometer measurements, in environments without known fiducials. This paper targets applications where external positions references such as global positioning are not available, and focuses on the use of small and inexpensive inertial sensors, for applications where weight and cost requirements preclude the use of precision inertial navigation systems. We present two algorithms for estimating sensor motion from image and inertial measurements. The first algorithm is a batch method, which produces estimates of the sensor motion, scene structure, and other unknowns using measurements from the entire observation sequence simultaneously. The second algorithm recovers sensor motion, scene structure, and other parameters recursively, and is suitable for use with long or “infinite” sequences, in which no feature is always visible. We evaluate the accuracy of the algorithms and their sensitivity to their estimation parameters using a sequence of four experiments. These experiments focus on cases where estimates from image or inertial measurements alone are poor, on the relative advantage of using inertial measurements and omni directional images, and on long sequences in which the percentage of the image sequence in which individual features are visible is low.

The power of comparative reasoning

Rank correlation measures are known for their resilience to perturbations in numeric values and are widely used in many evaluation metrics. Such ordinal measures have rarely been applied in treatment of numeric features as a representational transformation. We emphasize the benefits of ordinal representations of input features both theoretically and empirically. We present a family of algorithms for computing ordinal embeddings based on partial order statistics. Apart from having the stability benefits of ordinal measures, these embeddings are highly nonlinear, giving rise to sparse feature spaces highly favored by several machine learning methods. These embeddings are deterministic, data independent and by virtue of being based on partial order statistics, add another degree of resilience to noise. These machine-learning-free methods when applied to the task of fast similarity search outperform state-of-the-art machine learning methods with complex optimization setups. For solving classification problems, the embeddings provide a nonlinear transformation resulting in sparse binary codes that are well-suited for a large class of machine learning algorithms. These methods show significant improvement on VOC 2010 using simple linear classifiers which can be trained quickly. Our method can be extended to the case of polynomial kernels, while permitting very efficient computation. Further, since the popular Min Hash algorithm is a special case of our method, we demonstrate an efficient scheme for computing Min Hash on conjunctions of binary features. The actual method can be implemented in about 10 lines of code in most languages (2 lines in MAT-LAB), and does not require any data-driven optimization.

Precise omnidirectional camera calibration

Recent omnidirectional camera designs aim a conventional camera at a mirror that expands the cameras field of view. This wide view is ideal for three-dimensional vision tasks such as motion estimation and obstacle detection, but these applications require an accurate model of the imaging process. We present a full model of the imaging process, which includes the rotation and translation between the camera and mirror, and an algorithm that determines this relative position from observations of known points in a single image. We present tests of the model and of the calibration procedure for various amounts of misalignment between the mirror and camera. These tests show that the algorithm recovers the correct relative position, and that by using the full model, accurate shape-from-motion and stereo matching are possible even if the camera and mirror are severely misaligned.

Optimal motion estimation from visual and inertial measurements

Cameras and inertial sensors are good candidates to be deployed together for autonomous vehicle motion estimation, since each can be used to resolve the ambiguities in the estimated motion that results from using the other modality alone. We present an algorithm that computes optimal vehicle motion estimates by considering all of the measurements from a camera, rate gyro, and accelerometer simultaneously. Such optimal estimates are useful in their own right, and as a gold standard for the comparison of online algorithms. By comparing the motions estimated using visual and inertial measurements, visual measurements only, and inertial measurements only against ground truth, we show that using image and inertial data together can produce highly accurate estimates even when the results produced by each modality alone are very poor Our test datasets include both conventional and omnidirectional image sequences, and an image sequence with a high percentage of missing data.

General and nested Wiberg minimization

Wiberg matrix factorization breaks a matrix Y into low-rank factors U and V by solving for V in closed form given U, linearizing V (U) about U, and iteratively minimizing ∥Y-UV (U)∥with respect to U only. This approach factors the matrix while effectively removing V from the minimization. Recently Eriksson and van den Hengel extended this approach to L1, minimizing ∥Y-UV (U)∥1. We generalize their approach beyond factorization to minimize an arbitrary function that is nonlinear in each of two sets of variables. We demonstrate the idea with a practical Wiberg algorithm for L1 bundle adjustment. We also show that one Wiberg minimization can be nested inside another, effectively removing two of three sets of variables from a minimization. We demonstrate this idea with a nested Wiberg algorithm for L1 projective bundle adjustment, solving for camera matrices, points, and projective depths. We also revisit L1 factorization, giving a greatly simplified presentation of Wiberg L1 factorization, and presenting a successive linear programming factorization algorithm. Successive linear programming outperforms L1 Wiberg for most large inputs, establishing a new state-of-the-art for for those cases.

Extending shape-from-motion to noncentral onmidirectional cameras

Algorithms for shape-from-motion simultaneously estimate the camera motion and scene structure. When extended to omnidirectional cameras, shape-from-motion algorithms are likely to provide robust motion estimates, in particular, because of the cameras wide field of view. In this paper, we describe both batch and online shape-from-motion algorithms for omnidirectional cameras, and a precise calibration technique that improves the accuracy of both methods. The shape-from-motion and calibration methods are general, and they handle a wide variety of omnidirectional camera geometries. In particular, the methods do not require that the camera-mirror combination have a single center of projection. We describe a noncentral camera that we have developed, and show experimentally that combining shape-from-motion with this design produces highly accurate motion estimates.

An empirical comparison of methods for image-based motion estimation

This paper presents a comparison between methods that estimate motion of a camera from a sequence of video images. We implemented two methods: a homography based method that assumes planar environments; and shape-from-motion, a general method that can deal with a fully three dimensional world. Both methods were formulated in an iterative, online form to produce estimates of camera motion. We discuss a trade-off in accuracy and run time efficiency based on experimental results for these two general methods in relation to ground truth. We show how a variation of the homography method can produce accurate results in some cases when the environment is non-planar with low computational cost.

Reducing Web latency with hierarchical cache-based prefetching

Proxy caches have become a central mechanism for reducing the latency of Web document retrieval. While caching alone reduces latency for previously requested documents, Web document prefetching could mask latency for previously unseen, but correctly predicted requests. We describe a prefetching algorithm suitable for use in a network of hierarchical Web caches; this algorithm observes requests to a cache and its ancestors, and initiates prefetching for predicted future requests if prefetching is likely to reduce the overall latency seen by the caches clients. We introduce a novel cost-benefit model that allows us to judge the value of any cached or prefetched document, which we use to state a formal prefetching policy. Extensive simulations were run to judge the improvements offered by prefetching, and our approach is quantitatively compared to the method currently in use.

Recent Results in Extensions to Simultaneous Localization and Mapping

We report experimental results with bearings-only and range-only Simultaneous Localization and Mapping (SLAM). In the former case, we give the initial results from a new method that extends optimal shape-from-motion to incorporate angular rate and linear acceleration data. In the latter case, we have formulated a version of the SLAM problem that presumes a moving sensor able to measure only range to landmarks in the environment. Experimental results for both are presented.

General and nested wiberg minimization: L 2 and maximum likelihood

Wiberg matrix factorization breaks a matrix Y into low-rank factors U and V by solving for V in closed form given U, linearizing V(U) about U, and iteratively minimizing ||Y−UV(U)||2 with respect to U only. This approach factors the matrix while effectively removing V from the minimization. We generalize the Wiberg approach beyond factorization to minimize an arbitrary function that is nonlinear in each of two sets of variables. In this paper we focus on the case of L2 minimization and maximum likelihood estimation (MLE), presenting an L2 Wiberg bundle adjustment algorithm and a Wiberg MLE algorithm for Poisson matrix factorization. We also show that one Wiberg minimization can be nested inside another, effectively removing two of three sets of variables from a minimization. We demonstrate this idea with a nested Wiberg algorithm for L2 projective bundle adjustment, solving for camera matrices, points, and projective depths.