Peter Reinartz

Mutual-Information-Based Registration of TerraSAR-X and Ikonos Imagery in Urban Areas

The launch of high-resolution remote sensing satellites like TerraSAR-X, WorldView, and Ikonos has benefited the combined application of synthetic aperture radar (SAR) and optical imageries tremendously. Specifically, in case of natural calamities or disasters, decision makers can now easily use an old archived optical with a newly acquired (postdisaster) SAR image. Although the latest satellites provide the end user already georeferenced and orthorectified data products, still, registration differences exist between different data sets. These differences need to be taken care of through quick automated registration techniques before using the images in different applications. Specifically, mutual information (MI) has been utilized for the intricate SAR-optical registration problem. The computation of this metric involves estimating the joint histogram directly from image intensity values, which might have been generated from different sensor geometries and/or modalities (e.g., SAR and optical). Satellites carrying high-resolution remote sensing sensors like TerraSAR-X and Ikonos generate enormous data volume along with fine Earth observation details that might lead to failure of MI to detect correct registration parameters. In this paper, a solely histogram-based method to achieve automatic registration within TerraSAR-X and Ikonos images acquired specifically over urban areas is analyzed. Taking future sensors into a perspective, techniques like compression and segmentation for handling the enormous data volume and incompatible radiometry generated due to different SAR-optical image acquisition characteristics have been rightfully analyzed. The findings indicate that the proposed method is successful in estimating large global shifts followed by a fine refinement of registration parameters for high-resolution images acquired over dense urban areas.

Building Change Detection Based on Satellite Stereo Imagery and Digital Surface Models

Building change detection is a major issue for urban area monitoring. Due to different imaging conditions and sensor parameters, 2-D information delivered by satellite images from different dates is often not sufficient when dealing with building changes. Moreover, due to the similar spectral characteristics, it is often difficult to distinguish buildings from other man-made constructions, like roads and bridges, during the change detection procedure. Therefore, stereo imagery is of importance to provide the height component which is very helpful in analyzing 3-D building changes. In this paper, we propose a change detection method based on stereo imagery and digital surface models (DSMs) generated with stereo matching methodology and provide a solution by the joint use of height changes and Kullback-Leibler divergence similarity measure between the original images. The Dempster-Shafer fusion theory is adopted to combine these two change indicators to improve the accuracy. In addition, vegetation and shadow classifications are used as no-building change indicators for refining the change detection results. In the end, an object-based building extraction method based on shape features is performed. For evaluation purpose, the proposed method is applied in two test areas, one is in an industrial area in Korea with stereo imagery from the same sensor and the other represents a dense urban area in Germany using stereo imagery from different sensors with different resolutions. Our experimental results confirm the efficiency and high accuracy of the proposed methodology even for different kinds and combinations of stereo images and consequently different DSM qualities.

Applicability of the SIFT operator to geometric SAR image registration

The SIFT operators success for computer vision applications makes it an attractive alternative to the intricate feature based SAR image registration problem. The SIFT operator processing chain is capable of detecting and matching scale and affine invariant features. For SAR images, the operator is expected to detect stable features at lower scales where speckle influence diminishes. To adapt the operator performance to SAR images we analyse the impact of image filtering and of skipping features detected at the highest scales. We present our analysis based on multisensor, multitemporal and different viewpoint SAR images. The operator shows potential to become a robust alternative for point feature based registration of SAR images as subpixel registration consistency was achieved for most of the tested datasets. Our findings indicate that operator performance in terms of repeatability and matching capability is affected by an increase in acquisition differences within the imagery. We also show that the proposed adaptations result in a significant speed-up compared to the original SIFT operator.

Adaptive Shadow Detection Using a Blackbody Radiator Model

The application potential of remotely sensed optical imagery is boosted through the increase in spatial resolution, and new analysis, interpretation, classification, and change detection methods are developed. Together with all the advantages, shadows are more present in such images, particularly in urban areas. This may lead to errors during data processing. The task of automatic shadow detection is still a current research topic. Since image acquisition is influenced by many factors such as sensor type, sun elevation and acquisition time, geographical coordinates of the scene, conditions and contents of the atmosphere, etc., the acquired imagery has highly varying intensity and spectral characteristics. The variance of these characteristics often leads to errors, using standard shadow detection methods. Moreover, for some scenes, these methods are inapplicable. In this paper, we present an alternative robust method for shadow detection. The method is based on the physical properties of a blackbody radiator. Instead of static methods, this method adaptively calculates the parameters for a particular scene and allows one to work with many different sensors and images obtained with different illumination conditions. Experimental assessment illustrates significant improvement for shadow detection on typical multispectral sensors in comparison to other shadow detection methods. Examples, as well as quantitative assessment of the results, are presented for Landsat-7 Enhanced Thematic Mapper Plus, IKONOS, WorldView-2, and the German Aerospace Center (DLR) 3K Camera airborne system.

Performance Evaluation for 3-D City Model Generation of Six Different DSMs From Air- and Spaceborne Sensors

Since remote sensing provides more and more sensors and techniques to accumulate data on urban regions, three-dimensional representations of these complex environments gained much interest for various applications. In order to obtain three-dimensional representations, one of the most practical ways is to generate Digital Surface Models (DSMs) using very high resolution remotely sensed images from two or more viewing directions, or by using LIDAR sensors. Due to occlusions, matching errors and interpolation techniques these DSMs do not exhibit completely steep walls, and in order to obtain real three-dimensional urban models including objects like buildings from these DSMs, advanced methods are needed. A novel approach based on building shape detection, height estimation, and rooftop reconstruction is proposed to achieve realistic three-dimensional building representations. Our automatic approach consists of three main modules as; detection of complex building shapes, understanding rooftop type, and three-dimensional building model reconstruction based on detected shape and rooftop type. Besides the development of the methodology, the goal is to investigate the applicability and accuracy which can be accomplished in this context for different stereo sensor data. We use DSMs of Munich city which are obtained from different satellite (Cartosat-1, Ikonos, WorldView-2) and airborne sensors (3K camera, HRSC, and LIDAR). The paper later focuses on a quantitative comparisons of the outputs from the different multi-view sensors for a better understanding of qualities, capabilities and possibilities for applications. Results look very promising even for the DSMs derived from satellite data.

Modifications in the SIFT operator for effective SAR image matching

With the increasing availability and rapidly improving the spatial resolution of synthetic aperture radar (SAR) images from the latest and future satellites like TerraSAR-X and TanDEM-X, their applicability in remote sensing applications is set to be paramount. Considering challenges in the field of point feature-based multisensor/multimodal SAR image matching/registration and advancements in the field of computer vision, we extend the applicability of the scale invariant feature transform (SIFT) operator for SAR images. In this article, we have analysed the feature detection, identification and matching steps of the original SIFT processing chain. We implement steps to counter the speckle influence, which deteriorates the SIFT operator performance for SAR images. In feature identification, we evaluate different local gradient estimating techniques and highlight the fact that giving up the SIFTs rotation invariance characteristic increases the potential number of matches when the multiple SAR images from different sensors have been acquired with the same geometrical acquisition parameters. In the feature matching stage, we propose to assist the standard SIFT matching scheme to utilise the SIFT operator capability for effective results in challenging SAR image matching scenarios. The results obtained for SAR images acquired by different sensors using different incidence angles and orbiting directions over both rural and semi urban land cover, highlight the SIFT operators capability for point feature matching in SAR imagery.

Noise Reduction in Hyperspectral Images Through Spectral Unmixing

Spectral unmixing and denoising of hyperspectral images have always been regarded as separate problems. By considering the physical properties of a mixed spectrum, this letter introduces unmixing-based denoising, a supervised methodology representing any pixel as a linear combination of reference spectra in a hyperspectral scene. Such spectra are related to some classes of interest, and exhibit negligible noise influences, as they are averaged over areas for which ground truth is available. After the unmixing process, the residual vector is mostly composed by the contributions of uninteresting materials, unwanted atmospheric influences and sensor-induced noise, and is thus ignored in the reconstruction of each spectrum. The proposed method, in spite of its simplicity, is able to remove noise effectively for spectral bands with both low and high signal-to-noise ratio. Experiments show that this method could be used to retrieve spectral information from corrupted bands, such as the ones placed at the edge between ultraviolet and visible light frequencies, which are usually discarded in practical applications. The proposed method achieves better results in terms of visual quality in comparison to competitors, if the mean squared error is kept constant. This leads to questioning the validity of mean squared error as a predictor for image quality in remote sensing applications.

An Operational System for Estimating Road Traffic Information from Aerial Images

Given that ground stationary infrastructures for traffic monitoring are barely able to handle everyday traffic volumes, there is a risk that they could fail altogether in situations arising from mass events or disasters. In this work, we present an alternative approach for traffic monitoring during disaster and mass events, which is based on an airborne optical sensor system. With this system, optical image sequences are automatically examined on board an aircraft to estimate road traffic information, such as vehicle positions, velocities and driving directions. The traffic information, estimated in real time on board, is immediately downlinked to a ground station. The airborne sensor system consists of a three-head camera system, a real-time-capable GPS/INS unit, five industrial PCs and a downlink unit. The processing chain for automatic extraction of traffic information contains modules for the synchronization of image and navigation data streams, orthorectification and vehicle detection and tracking modules. The vehicle detector is based on a combination of AdaBoost and support vector machine classifiers. Vehicle tracking relies on shape-based matching operators. The processing chain is evaluated on a large number of image sequences recorded during several campaigns, and the data quality is compared to that obtained from induction loops. In summary, we can conclude that the achieved overall quality of the traffic data extracted by the airborne system is in the range of 68% and 81%. Thus, it is comparable to data obtained from stationary ground sensor networks.

Image acquisition geometry analysis for the fusion of optical and radar remote sensing data

Fusion of optical and radar remote sensing data is becoming an actual topic of discussion recently in various application areas though the results are not always satisfactory. In this article, we analyse some disturbing aspects of fusing orthoimages from sensors having different acquisition geometries. These aspects arise due to errors in digital elevation models (DEM), used for image orthorectification, and the existence of 3-D objects in the scene which are not accounted in the DEM. We analyse how these effects influence the ground displacement in orthoimages produced from optical and radar data. Further, we propose sensor formations with acquisition geometry parameters which allow to minimise or compensate for ground displacements in different orthoimages due to the above-mentioned effects and to produce good prerequisites for the following fusion for specific application areas, e.g. matching, filling data gaps, classification, etc. To demonstrate the potential of the proposed approach, two pairs of optical–radar data were acquired over the urban area–Munich City, Germany. The first collection of WorldView-1 and TerraSAR-X (TS-X) data followed the proposed recommendations for acquisition geometry parameters, whereas the second collection of IKONOS and TS-X data was acquired with accidental parameters. The experiment confirmed our ideas fully. Moreover, it opens new possibilities for optical and radar image fusion.

Building Reconstruction Using DSM and Orthorectified Images

High resolution Digital Surface Models (DSMs) produced from airborne laser-scanning or stereo satellite images provide a very useful source of information for automated 3D building reconstruction. In this paper an investigation is reported about extraction of 3D building models from high resolution DSMs and orthorectified images produced from Worldview-2 stereo satellite imagery. The focus is on the generation of 3D models of parametric building roofs, which is the basis for creating Level Of Detail 2 (LOD2) according to the CityGML standard. In particular the building blocks containing several connected buildings with tilted roofs are investigated and the potentials and limitations of the modeling approach are discussed. The edge information extracted from orthorectified image has been employed as additional source of information in 3D reconstruction algorithm. A model driven approach based on the analysis of the 3D points of DSMs in a 2D projection plane is proposed. Accordingly, a building block is divided into smaller parts according to the direction and number of existing ridge lines for parametric building reconstruction. The 3D model is derived for each building part, and finally, a complete parametric model is formed by merging the 3D models of the individual building parts and adjusting the nodes after the merging step. For the remaining building parts that do not contain ridge lines, a prismatic model using polygon approximation of the corresponding boundary pixels is derived and merged to the parametric models to shape the final model of the building. A qualitative and quantitative assessment of the proposed method for the automatic reconstruction of buildings with parametric roofs is then provided by comparing the final model with the existing surface model as well as some field measurements.