Arnold Irschara

From structure-from-motion point clouds to fast location recognition

Efficient view registration with respect to a given 3D reconstruction has many applications like inside-out tracking in indoor and outdoor environments, and geo-locating images from large photo collections. We present a fast location recognition technique based on structure from motion point clouds. Vocabulary tree-based indexing of features directly returns relevant fragments of 3D models instead of documents from the images database. Additionally, we propose a compressed 3D scene representation which improves recognition rates while simultaneously reducing the computation time and the memory consumption. The design of our method is based on algorithms that efficiently utilize modern graphics processing units to deliver real-time performance for view registration. We demonstrate the approach by matching hand-held outdoor videos to known 3D urban models, and by registering images from online photo collections to the corresponding landmarks.

Wide area localization on mobile phones

We present a fast and memory efficient method for localizing a mobile users 6DOF pose from a single camera image. Our approach registers a view with respect to a sparse 3D point reconstruction. The 3D point dataset is partitioned into pieces based on visibility constraints and occlusion culling, making it scalable and efficient to handle. Starting with a coarse guess, our system only considers features that can be seen from the users position. Our method is resource efficient, usually requiring only a few megabytes of memory, thereby making it feasible to run on low-end devices such as mobile phones. At the same time it is fast enough to give instant results on this device class.

Towards Wiki-based Dense City Modeling

This work reports on the advances and on the current status of a terrestrial city modeling approach, which uses images contributed by end-users as input. Hence, the Wiki principle well known from textual knowledge databases is transferred to the goal of incrementally building a virtual representation of the occupied habitat. In order to achieve this objective, many state-of-the-art computer vision methods must be applied and modified according to this task. We describe the utilized 3D vision methods and show initial results obtained from the current image database acquired by in-house participants.

Natural landmark-based monocular localization for MAVs

Highly accurate localization of a micro aerial vehicle (MAV) with respect to a scene is important for a wide range of applications, in particular surveillance and inspection. Most existing approaches to visual localization focus on indoor environments, while such tasks require outdoor navigation. Within this work, we introduce a novel algorithm for monocular visual localization for MAVs based on the concept of virtual views in 3D space. Under the assumption that significant parts of the scene do not alter their geometry and serve as natural landmarks, the accuracy of our visual approach outperforms consumer grade GPS systems. In an experimental setup we compare our approach to a state-of-the-art visual SLAM algorithm and evaluate the performance by geometric validation from an observers view. As our method directly allows global registration, it is neither prone to drift nor bias. This makes it well suited for long-term autonomous navigation.

What can missing correspondences tell us about 3D structure and motion

Practically all existing approaches to structure and motion computation use only positive image correspondences to verify the camera pose hypotheses. Incorrect epipolar geometries are solely detected by identifying outliers among the found correspondences. Ambiguous patterns in the images are often incorrectly handled by these standard methods. In this work we propose two approaches to overcome such problems. First, we apply non-monotone reasoning on view triplets using a Bayesian formulation. In contrast to two-view epipolar geometry, image triplets allow the prediction of features in the third image. Absence of these features (i.e. missing correspondences) enables additional inference about the view triplet. Furthermore, we integrate these view triplet handling into an incremental procedure for structure and motion computation. Thus, our approach is able to refine the maintained 3D structure when additional image data is provided.

Efficient structure from motion with weak position and orientation priors

In this paper we present an approach that leverages prior information from global positioning systems and inertial measurement units to speedup structure from motion computation. We propose a view selection strategy that advances vocabulary tree based coarse matching by also considering the geometric configuration between weakly oriented images. Furthermore, we introduce a fast and scalable reconstruction approach that relies on global rotation registration and robust bundle adjustment. Real world experiments are performed using data acquired by a micro aerial vehicle attached with GPS/INS sensors. Our proposed algorithm achieves orientation results that are sub-pixel accurate and the precision is on a par with results from incremental structure from motion approaches. Moreover, the method is scalable and computationally more efficient than previous approaches.

Multi-camera multi-object tracking by robust hough-based homography projections

Recently, several approaches have been introduced for incorporating the information from multiple cameras to increase the robustness of tracking. This allows to handle problems of mutually occluding objects - a reasonable scenario for many tasks such as visual surveillance or sports analysis. However, these methods often ignore problems such as inaccurate geometric constraints and violated geometric assumptions, requiring complex methods to resolve the resulting errors. In this paper, we introduce a new multiple camera tracking approach that inherently avoids these problems. We build on the ideas of generalized Hough voting and extend it to the multiple camera domain. This offers the following advantages: we reduce the amount of data in voting and are robust to projection errors. Moreover, we show that using additional geometric information can help to train more specific classifiers drastically improving the tracking performance. We confirm these findings by comparing our approach to existing (multi-camera) tracking methods.

Large-scale, dense city reconstruction from user-contributed photos

The goal of our work is to incrementally reconstruct terrestrial city models from standard digital camera images contributed by multiple users. Hence, the Wiki principle well known from textual knowledge databases is transferred to 3D computer vision. Many state-of-the-art computer vision methods must be applied and modified according to the changing requirements. We describe the utilized 3D vision methods in detail and show results obtained from the current image databases Vienna and Graz acquired by in-house participants. The reconstructions are all maintained in a global database and comprise thousands of photographs.

Automatic alignment of 3D reconstructions using a Digital Surface Model

We present a novel technique for the automatic alignment of Structure from Motion (SfM) models, acquired at ground level or by micro aerial vehicles, to an overhead Digital Surface Model (DSM) using GPS information. An additional refinement step based on the correlation of the DSM height map with the model height map corrects for the GPS localization uncertainties and results in precisely aligned models. Our approach successfully handles cases where previous methods had problems, including objects on the ground, unoccupied space, and models covering a small area. We conclude our work by presenting several applications of our approach, namely the fusion of detailed SfM model information into the original DSM, season-invariant matching using aligned models, and alignment for providing context in visualization.

Aerial Computer Vision for a 3D Virtual Habitat

An Internet-embedded 3D model of a human habitat is feasible and useful. In lieu of a 2D Earth map, the authors describe a 3D model with human-scale objects in urban spaces and inside buildings. Here, they focus on information from aerial imagery.