Jan Dirk Wegner

Building Detection From One Orthophoto and High-Resolution InSAR Data Using Conditional Random Fields

Todays airborne SAR sensors provide geometric resolution in the order well below half a meter. Many features of urban objects become visible in such data. However, layover and occlusion issues inevitably arise in urban areas complicating automated object detection. In order to support interpretation, SAR data may be analyzed using complementary information from maps or optical imagery. In this paper, an approach for building detection in urban areas based on object features extracted from high-resolution interferometric SAR (InSAR) data and one orthophoto is presented. Features describing local evidence as well as context information are used. Buildings are detected by classification of those feature vectors within a Conditional Random Field (CRF) framework. Although as graphical model similar to Markov Random Fields (MRF), CRFs have the advantage of incorporating global context information, of relaxing the conditional independence assumption between features, and of a more general integration of observations. We show that, first, CRFs perform well in comparison to Maximum Likelihood classifiers and MRFs. Second, the combined use of optical and InSAR features may improve detection results.

A Higher-Order CRF Model for Road Network Extraction

The aim of this work is to extract the road network from aerial images. What makes the problem challenging is the complex structure of the prior: roads form a connected network of smooth, thin segments which meet at junctions and crossings. This type of a-priori knowledge is more difficult to turn into a tractable model than standard smoothness or co-occurrence assumptions. We develop a novel CRF formulation for road labeling, in which the prior is represented by higher-order cliques that connect sets of super pixels along straight line segments. These long-range cliques have asymmetric PN-potentials, which express a preference to assign all rather than just some of their constituent super pixels to the road class. Thus, the road likelihood is amplified for thin chains of super pixels, while the CRF is still amenable to optimization with graph cuts. Since the number of such cliques of arbitrary length is huge, we furthermore propose a sampling scheme which concentrates on those cliques which are most relevant for the optimization. In experiments on two different databases the model significantly improves both the per-pixel accuracy and the topological correctness of the extracted roads, and outperforms both a simple smoothness prior and heuristic rule-based road completion.

Features, Color Spaces, and Boosting: New Insights on Semantic Classification of Remote Sensing Images

A major yet largely unsolved problem in the semantic classification of very high resolution remote sensing images is the design and selection of appropriate features. At a ground sampling distance below half a meter, fine-grained texture details of objects emerge and lead to a large intraclass variability while generally keeping the between-class variability at a low level. Usually, the user makes an educated guess on what features seem to appropriately capture characteristic object class patterns. Here, we propose to avoid manual feature selection and let a boosting classifier choose optimal features from a vast Randomized Quasi-Exhaustive (RQE) set of feature candidates directly during training. This RQE feature set consists of a multitude of very simple features that are computed efficiently via integral images inside a sliding window. This simple but comprehensive feature candidate set enables the boosting classifier to assemble the most discriminative textures at different scale levels to classify a small number of broad urban land-cover classes. We do an extensive evaluation on several data sets and compare performance against multiple feature extraction baselines in different color spaces. In addition, we verify experimentally if we gain any classification accuracy if moving from boosting stumps to trees. Cross-validation minimizes the possible bias caused by specific training/testing setups. It turns out that boosting in combination with the proposed RQE feature set outperforms all baseline features while still remaining computationally efficient. Particularly boosting trees (instead of stumps) captures class patterns so well that results suggest to completely leave feature selection to the classifier.

Conditional random fields for urban scene classification with full waveform LiDAR data

We propose a context-based classification method for point clouds acquired by full waveform airborne laser scanners. As these devices provide a higher point density and additional information like echo width or type of return, an accurate distinction of several object classes is possible. However, especially in dense urban areas correct labelling is a challenging task. Therefore, we incorporate context knowledge by using Conditional Random Fields. Typical object structures are learned in a training step and improve the results of the point-based classification process. We validate our approach with two real-world datasets and by a comparison to Support Vector Machines and Markov Random Fields.

Large-Scale Semantic 3D Reconstruction: An Adaptive Multi-resolution Model for Multi-class Volumetric Labeling

We propose an adaptive multi-resolution formulation of semantic 3D reconstruction. Given a set of images of a scene, semantic 3D reconstruction aims to densely reconstruct both the 3D shape of the scene and a segmentation into semantic object classes. Jointly reasoning about shape and class allows one to take into account class-specific shape priors (e.g., building walls should be smooth and vertical, and vice versa smooth, vertical surfaces are likely to be building walls), leading to improved reconstruction results. So far, semantic 3D reconstruction methods have been limited to small scenes and low resolution, because of their large memory footprint and computational cost. To scale them up to large scenes, we propose a hierarchical scheme which refines the reconstruction only in regions that are likely to contain a surface, exploiting the fact that both high spatial resolution and high numerical precision are only required in those regions. Our scheme amounts to solving a sequence of convex optimizations while progressively removing constraints, in such a way that the energy, in each iteration, is the tightest possible approximation of the underlying energy at full resolution. In our experiments the method saves up to 98% memory and 95% computation time, without any loss of accuracy.

Mind the Gap: Modeling Local and Global Context in (Road) Networks

We propose a method to label roads in aerial images and extract a topologically correct road network. Three factors make road extraction difficult: (i) high intra-class variability due to clutter like cars, markings, shadows on the roads; (ii) low inter-class variability, because some non-road structures are made of similar materials; and (iii) most importantly, a complex structural prior: roads form a connected network of thin segments, with slowly changing width and curvature, often bordered by buildings, etc. We model this rich, but complicated contextual information at two levels. Locally, the context and layout of roads is learned implicitly, by including multi-scale appearance information from a large neighborhood in the per-pixel classifier. Globally, the network structure is enforced explicitly: we first detect promising stretches of road via shortest-path search on the per-pixel evidence, and then select pixels on an optimal subset of these paths by energy minimization in a CRF, where each putative path forms a higher-order clique. The model outperforms several baselines on two challenging data sets, both in terms of precision/recall and w.r.t. topological correctness.

Building Reconstruction from Multi-aspect InSAR Data

Modern space borne SAR sensors like TerraSAR-X and Cosmo-SkyMed provide geometric ground resolution of one meter. Airborne sensors (PAMIR [Brenner and Ender 2006], SETHI [Dreuillet et al. 2008]) achieve even higher resolution. In data of such kind, man-made structures in urban areas become visible in detail independently from daylight or cloud coverage. Typical objects of interest for both civil and military applications are buildings, bridges, and roads. However, phenomena due to the side-looking scene illumination of the SAR sensor complicate interpretability (Schreier 1993). Layover, foreshortening, shadowing, total reflection, and multi-bounce scattering of the RADAR signal hamper manual and automatic analysis especially in dense urban areas with high buildings. Such drawbacks may partly be overcome using additional information from, for example topographic maps, optical imagery (see corresponding chapter in this book), or SAR acquisitions from multiple aspects.

Contour Detection in Unstructured 3D Point Clouds

We describe a method to automatically detect contours, i.e. lines along which the surface orientation sharply changes, in large-scale outdoor point clouds. Contours are important intermediate features for structuring point clouds and converting them into high-quality surface or solid models, and are extensively used in graphics and mapping applications. Yet, detecting them in unstructured, inhomogeneous point clouds turns out to be surprisingly difficult, and existing line detection algorithms largely fail. We approach contour extraction as a two-stage discriminative learning problem. In the first stage, a contour score for each individual point is predicted with a binary classifier, using a set of features extracted from the points neighborhood. The contour scores serve as a basis to construct an overcomplete graph of candidate contours. The second stage selects an optimal set of contours from the candidates. This amounts to a further binary classification in a higher-order MRF, whose cliques encode a preference for connected contours and penalize loose ends. The method can handle point clouds > 107 points in a couple of minutes, and vastly outperforms a baseline that performs Canny-style edge detection on a range image representation of the point cloud.

Combining High-Resolution Optical and InSAR Features for Height Estimation of Buildings With Flat Roofs

In this paper, we contribute to answer the question: How accurately can we estimate heights of buildings with flat roofs given one high-resolution single-pass interferometric synthetic aperture radar (InSAR) image pair and one aerial orthophoto? What makes this problem challenging are the different sensor geometries and the sound stochastic combination of all available elevation cues. We revisit already existing methods and develop novel approaches to determine building heights. A rigorous stochastic approach based on least squares adjustment with functionally dependent parameters is introduced to combine all height measurements per building to one robust height estimate. Observation accuracies of the stochastic model are either taken from the literature or estimated empirically. A major benefit of adjustment is that it delivers a posterior standard deviation per height, which can be interpreted as a precision indicator and is of high relevance for practical applications. Estimated heights of an urban scene are compared to ground truth acquired with airborne laser scanning, allowing us to assess height accuracies that can be achieved under nearly optimal conditions. We conduct statistical tests that validate our model and show that a weighted combination of optical and synthetic aperture radar (SAR) data with least squares adjustment delivers reliable height estimates with meter accuracy for flat-roofed buildings. Additionally, we empirically estimate a confidence interval of the estimated heights that directly tells the user the security margin to be included, for example, in case of building evacuations for an anticipated flooding event, under the condition that the data and model have the same specifications as in this paper.

Building extraction in urban scenes from high-resolution InSAR data and optical imagery

Modern space borne SAR sensors provide geometric resolution of one meter, airborne systems even higher. In data of this kind many features of urban objects become visible, which were beyond the scope of radar remote sensing only a few years ago. However, layover and occlusion issues inevitably arise in undulated terrain and urban areas because of the side-looking SAR sensor principle. In order to support interpretation, SAR data are often analyzed using additional complementary information provided by maps or other remote sensing imagery. The focus of this paper is on building extraction in urban scenes by means of combined InSAR data and optical aerial imagery.