Shiyong Cui

Building Change Detection Based on Satellite Stereo Imagery and Digital Surface Models

Building change detection is a major issue for urban area monitoring. Due to different imaging conditions and sensor parameters, 2-D information delivered by satellite images from different dates is often not sufficient when dealing with building changes. Moreover, due to the similar spectral characteristics, it is often difficult to distinguish buildings from other man-made constructions, like roads and bridges, during the change detection procedure. Therefore, stereo imagery is of importance to provide the height component which is very helpful in analyzing 3-D building changes. In this paper, we propose a change detection method based on stereo imagery and digital surface models (DSMs) generated with stereo matching methodology and provide a solution by the joint use of height changes and Kullback-Leibler divergence similarity measure between the original images. The Dempster-Shafer fusion theory is adopted to combine these two change indicators to improve the accuracy. In addition, vegetation and shadow classifications are used as no-building change indicators for refining the change detection results. In the end, an object-based building extraction method based on shape features is performed. For evaluation purpose, the proposed method is applied in two test areas, one is in an industrial area in Korea with stereo imagery from the same sensor and the other represents a dense urban area in Germany using stereo imagery from different sensors with different resolutions. Our experimental results confirm the efficiency and high accuracy of the proposed methodology even for different kinds and combinations of stereo images and consequently different DSM qualities.

Ratio-Detector-Based Feature Extraction for Very High Resolution SAR Image Patch Indexing

With the advent of very high resolution (VHR) synthetic aperture radar (SAR) images, local content description is becoming a critical issue for indexing. Conventional SAR image analysis techniques, like segmentation and pixel-level classification, are likely to fail as high-level semantic description should be considered for better discrimination. Therefore, we propose to use image-patch-based analysis method for SAR image interpretation. Inspired by ratio edge detector, in this letter, a new feature extraction method represented by the mean ratios in different directions is proposed for VHR SAR image content characterization. Based on the mean ratio, two simple yet powerful and robust features are proposed for SAR image patch indexing. One is the bag-of-word model using not only the basic statistics, i.e., local mean and variance, but also the mean ratios in different directions. The second one is an adaptation of the Weber local descriptor to SAR images by substituting the gradient with the ratio of mean differences in vertical and horizontal directions. To evaluate the proposed features, image patch indexing based on active learning using a SAR image database consisting of high-resolution TerraSAR-X patches is performed. Comparison with the state-of-the-art features, particularly texture features, has shown improved performance for SAR image categorization.

Statistical Wavelet Subband Modeling for Multi-Temporal SAR Change Detection

In the context of multi-temporal SAR change detection for earth monitoring applications, one critical issue is to generate accurate change map. A common method to generate change map is to apply logarithm to the ratio image. However, due to the speckle effect and without consideration of contextual information, it is usually not efficient for accurate change detection. In this paper, an unsupervised change detection method in wavelet domain based on statistical wavelet subband modeling is proposed. The motivation is to capture textures efficiently in wavelet domain. Wavelet transform is applied to decompose the image into multiple scales and probability density function of the coefficient magnitudes of each subband assumed to be Generalized Gaussian Distribution (GGD) and Generalized Gamma Distribution (G) are obtained by fast parameter estimation. Closed-form expression of Kullback-Leibler divergence between two corresponding subbands of the same scale is computed and used to generate the change map. This approach is comprehensively evaluated and compared using different parameter setting, different scales, window sizes and estimators. The proposed SAR change detection in wavelet domain shows promising results as texture can be better characterized in wavelet domain than in spatial domain. Through this study, we conclude that the accuracy depends heavily on the estimation methods although the model is important. Both parameter estimation for GGD based on shape equation and parameter estimation for G using method of log-cumulants (MoLC) in wavelet domain performs quite well.

Complex building description and extraction based on Hough transformation and cycle detection

A simple but robust approach for complex building description and extraction from high-resolution remotely sensed imagery based on graph-based shape representation is proposed. Classical approaches for building extraction usually involve a complex grouping process of low-level primitive features and are not robust in the presence of noise. To overcome these drawbacks, this approach presents an efficient and robust solution by integrating edges and regions. First, a region segmentation method is applied to obtain the approximate shape of the building. Second, Hough transformation is employed to derive the two perpendicular line sets corresponding to the building boundary. Third, a subset of the intersectional nodes of the two line sets is utilized to construct a building structural graph, based on the analysis of grey value difference between the two sides of each line segment. Finally, a graph search algorithm is performed to retrieve all the cycles in the structural graph. The cycle corresponding to the building boundary is identified as the final building outline on the basis of its area. Two experiments were carried out to evaluate and validate this approach and experimental results confirm its effectiveness and robustness.

Pattern Retrieval in Large Image Databases Using Multiscale Coarse-to-Fine Cascaded Active Learning

Pattern retrieval is a fundamental challenge in machine learning but is often subject to the problem of gathering enough labeled examples of the target pattern, and also to the computational complexity inherent to the training and the evaluation of complex classifier functions on large databases. In this paper, we propose a hierarchical top-down processing scheme for pattern retrieval in high-volume high-resolution optical satellite image repositories. We learn via a multistage active learning process a cascade of classifiers working each at a certain scale on a patch-based representation of images. At each stage of the hierarchy, we seek to eliminate large parts of images considered as nonrelevant, the purpose being to set the focus at the finest scales on more promising and as spatially limited as possible areas. Our scheme is based on the fact that by reducing the size of the analysis window (i.e., the size of the patch), we better capture the properties of the targeted object. The cascaded hierarchy is introduced to compensate for the extra computational burden incurred by diminishing the size of the patch, which causes an explosion of the number of patches to process. Unlike most other retrieval methods, which require large training sets and costly offline training, we propose a cascaded active learning strategy to build a classifier at each level of the hierarchy, and we provide a new Multiple Instance Learning algorithm to propagate automatically the training examples from one level of the hierarchy to the other. Two study cases are performed for validation. The first is a test on a database of 61-cm resolution QuickBird panchromatic images and the second is an example of temporal pattern retrieval from a database of Synthetic Aperture Radar (SAR) image time series. These tests show that our method achieves a reduction in the number of computations of two orders of magnitude, while keeping the same accuracy level as recent state-of-the-art methods.

Information Content of Very-High-Resolution SAR Images: Semantics, Geospatial Context, and Ontologies

Currently, the amount of collected Earth Observation (EO) data is increasing considerably with a rate of several Terabytes of data per day. As a consequence of this increasing data volume, new concepts for exploration and information retrieval are urgently needed. To this end, we propose to explore satellite image data via an image information mining (IIM) approach in which the main steps are feature extraction, classification, semantic annotation, and interactive query processing. This leads to a new process chain and a robust taxonomy for the retrieved categories capitalizing on human interaction and judgment. We concentrated on land cover categories that can be retrieved from high-resolution synthetic aperture radar (SAR) images of the spaceborne TerraSAR-X instrument, where we annotated different urban areas all over the world and defined a taxonomy element for each prevailing surface cover category. The annotation resulted from a test dataset comprising more than 100 scenes covering diverse areas of Africa, Asia, Europe, the Middle East, and North and South America. The scenes were grouped into several collections with similar source areas and each collection was processed separately in order to discern regional characteristics. In the first processing step, each scene was tiled into patches. Then the features were extracted from each patch by a Gabor filter bank and a support vector machine with relevance feedback classifying the feature sets into user-oriented land cover categories. Finally, the categories were semantically annotated using Google Earth for ground truthing. The annotation followed a multilevel approach that allowed the fusion of information being visible on different resolution levels. The novelty of this paper lies in the fact that a semantic annotation was performed with a large number of high-resolution radar images that allowed the definition of more than 850 surface cover categories. This opens the way toward an automated identification and classification of urban areas, infrastructure (e.g., airports), geographic objects (e.g., mountains), industrial installations, military compounds, vegetation, and agriculture. Applications that may result from this work can be a semantic catalog of urban images to be used in crisis situations or after a disaster. In addition, the proposed taxonomies can become a basis for building a semantic catalog of satellite images. Finally, we defined four powerful types of high-level queries. Querying on such high levels provides new opportunities for users to search an image database for specific parameters or semantic relationships.

A Comparative Study of Bag-of-Words and Bag-of-Topics Models of EO Image Patches

The large volume of detailed land cover features, provided by high resolution Earth observation (EO) images, has attracted considerable interest in the discovery of these features by learning systems. In this letter, we perform latent Dirichlet allocation on the bag of words (BoW) representation of collections of EO image patches to discover their semantic-level features, the so-called topics. To assess the discovered topics, the images are represented based on the occurrence of different topics, called bag of topics (BoT). The value added by BoT to the BoW model of image patches is then measured based on existing human annotations of the data. In our experiments, we compare the classification accuracy results of BoT and BoW representations of two different remote sensing image data sets, a multispectral optical data set and a synthetic-aperture-radar data set. Experimental results demonstrate that BoT can provide a compact and semantically meaningful representation of data; it either causes no significant reduction in the classification accuracy or increases the accuracy by a sufficient number of topics.

A Comparative Study of Statistical Models for Multilook SAR Images

In this letter, we carry out a comparative study of statistical models for multilook synthetic aperture radar amplitude images. Ten state-of-the-art statistical models are selected for comparison. To achieve a fair evaluation, we estimate all model parameters using the method of log-cumulants and apply the method to an image pyramid with varying pixel spacing (and resolution). The pyramid is created by different image product generation options. In addition to pixel spacing and resolution, we also consider the homogeneity of a scene for performance evaluation and we apply three performance measures. Through this study, it was found out that some models perform well for all resolutions, while the performance of other models depends heavily on the image content.

Data Analytics for Rapid Mapping: Case Study of a Flooding Event in Germany and the Tsunami in Japan Using Very High Resolution SAR Images

In this paper, we present data analytics for a quantitative analysis in a rapid mapping scenario applied for damage assessment of the 2013 floods in Germany and the 2011 tsunami in Japan. These scenarios are created using preand postdisaster TerraSAR-X images and a semi-automated processing chain. All our dataset is tiled into patches and Gabor filters are applied as a primitive feature extraction method to each patch separately. A support vector machine with relevance feedback is implemented in order to group the features into categories. Once all categories are identified, these are semantically annotated using reference data as ground truth. In our investigation, nondamaged and damaged categories were retrieved with their specific taxonomies defined using our previous hierarchical annotation scheme. The classifier supports rapid mapping scenarios (e.g., floods in Germany and tsunami in Japan) and interactive mapping generation. The quantitative damages can be assessed by: 1) flooded agricultural areas (21.66% in the case of floods in Germany and 4.15% in the case of tsunami in Japan) and destroyed aquaculture (2.33% in the case of tsunami in Japan); 2) destroyed transportation infrastructures, such as airport (50% in case tsunami in Japan), bridges, and roads.; and 3) debris that appears in postdisaster images (3.81% in the case of tsunami after the aquaculture was destroyed). The first analysis envisages the floods of Elbe river in June 2013: 30% of the investigated area, about , including agricultural land, forest, river, and some residential and industrial areas close to the river, was covered by water. The second analysis, considering an area of affected by the tsunami, led us to conclude that 3 months after the tsunami, some of the categories returned to previous functionality-the airport, others return to partial functionality such as isolated residents, and some were totally destroyed-the aquaculture. The flooded area was about . The proposed approach goes beyond a simple annotation of the data and provides an intermediate product in order to produce a relevant visual analytics representation of the data. This makes it easier to compare datasets and different quantitative findings in a meaningful manner, accessible both to experts and regular users. Our paper presents an interactive and automatic, fast processing method applicable to large and complex datasets (such as image time series). In addition to enhancing the information content extraction (number of identified categories), this approach enables the discovery and analysis of these categories. The novelty of this paper resides in that this is the first time data analytics have been run on a large dataset and for different scenarios using a semiautomated processing chain.

A Benchmark Evaluation of Similarity Measures for Multitemporal SAR Image Change Detection

Synthetic aperture radar (SAR) image change detection is playing an important role in various Earth Observation (EO) applications. There exists a large number of different methods that have been proposed to address this issue. However, due to the fact that several kinds of changes with diverse characteristics can arise in SAR images, there is no consensus on their performances because most methods have been evaluated using different data sets, probably facing several kinds of changes, but without an in-depth analysis of the characteristics of SAR image changes. Therefore, two important problems arise. The first is what kind of change each approach can detect. The second is how much they can detect a kind of change. Although the importance to model any kind of changes has been realized, there is no principled methodology to carry out the analysis due to the difficulty in modeling various kinds of changes. In this paper, we propose a benchmark methodology to reach this goal by simulating selected kinds of changes in addition to using real data with changes. Six kinds of SAR changes for eight typical image categories are simulated, i.e., reflectivity changes, first-order, second-order, and higher order statistical changes, linear and nonlinear changes. Based on this methodology for change simulation, a comprehensive evaluation of information similarity measures is carried out. An explicit conclusion we have drawn from the evaluation is that the various methods behave very differently for all kinds of changes. We hope that this study will promote the advancement of this topic.