Corneliu Octavian Dumitru

Information Content of Very High Resolution SAR Images: Study of Feature Extraction and Imaging Parameters

In this paper, we propose to study the dependence of information extraction technique performance on synthetic aperture radar (SAR) imaging parameters and the selected primitive features (PFs). The evaluation is done on TerraSAR-X data, and the interpretation is realized automatically. In the first part of this paper (use case I), the following issues are analyzed: 1) finding the optimal TerraSAR-X products and their limits of variability and 2) retrieving the number of categories/classes that can be extracted from the TerraSAR-X images using the PFs (gray-level co-occurrence matrix, Gabor filters, quadrature mirror filters, and nonlinear short-time Fourier transform). In the second part of this paper (use case II), we investigate the invariance of the products with the orbit direction and incidence angle. On the one hand, the results show that using ascending looking is better than using descending looking with an average accuracy increase of 7%-8%, approximately. On the other hand, the classification accuracy for the incidence angle varies from a lower value of the incidence to an upper value of the incidence angle (depending on the sensor range) with 4%-5%. The test sites are Venice (Italy), Toulouse (France), Berlin (Germany), and Ottawa (Canada) and are covering as much as possible the huge diversity of modes, types, and geometric resolution configuration of the TerraSAR-X. For the evaluation of all these parameters (resolution, features, orbit looking, and incidence angle), the support-vector-machine classifier is considered. To evaluate the accuracy of the classification, the precision/recall metric is calculated. The first contribution of this paper is the evaluation of different PFs (proposed in the literature for different types of images) and adaptation of these for SAR images. These features are compared (based on the accuracy of the classification) for the first time for a multiresolution pyramid specially built for this purpose. During the evaluation, all the classes were annotated, and a semantic meaning was defined for each class. The second main contribution of this paper is the evaluation of the dependence on the patch size, orbit direction, and incidence angle of the TerraSAR-X. This type of evaluation has not been systematically investigated so far. For the evaluation of the optimal patch, two different patch sizes were defined, with the constrained that the size on ground needs to cover a minimum of one object (e.g., 200 × 200 m on ground). This patch size depends also on the parameters of the data such as resolution and pixel spacing. The investigation of orbit looking and incidence angle is very important for indexing large data sets that has a higher variability of these two parameters. These parameters influence the accuracy of the classification (e.g., if the incidence angle is closer to the lower bounds or closer to the upper bound of the satellite sensor range).

Ratio-Detector-Based Feature Extraction for Very High Resolution SAR Image Patch Indexing

With the advent of very high resolution (VHR) synthetic aperture radar (SAR) images, local content description is becoming a critical issue for indexing. Conventional SAR image analysis techniques, like segmentation and pixel-level classification, are likely to fail as high-level semantic description should be considered for better discrimination. Therefore, we propose to use image-patch-based analysis method for SAR image interpretation. Inspired by ratio edge detector, in this letter, a new feature extraction method represented by the mean ratios in different directions is proposed for VHR SAR image content characterization. Based on the mean ratio, two simple yet powerful and robust features are proposed for SAR image patch indexing. One is the bag-of-word model using not only the basic statistics, i.e., local mean and variance, but also the mean ratios in different directions. The second one is an adaptation of the Weber local descriptor to SAR images by substituting the gradient with the ratio of mean differences in vertical and horizontal directions. To evaluate the proposed features, image patch indexing based on active learning using a SAR image database consisting of high-resolution TerraSAR-X patches is performed. Comparison with the state-of-the-art features, particularly texture features, has shown improved performance for SAR image categorization.

Semantic annotation in earth observation based on active learning

As the data acquisition capabilities of earth observation (EO) satellites have been improved significantly, a large amount of high-resolution images are downlinked continuously to ground stations. The data volume increases rapidly beyond the users’ capability to access the information content of the data. Thus, interactive systems that allow fast indexing of high-resolution images based on image content are urgently needed. In this paper, we present an interactive learning system for semantic annotation and content mining at patch level. It mainly comprises four components: primitive feature extraction including both spatial and temporal features, relevance feedback based on active learning, a human machine communication (HMC) interface and data visualisation. To overcome the shortage of training samples and to speed up the convergence, active learning is employed in this system. Two core components of active learning are the classifier training using already labelled image patches, and the sample selection strategy which selects the most informative samples for manual labelling. These two components work alternatively, significantly reducing the labelling effort and achieving fast indexing. In addition, our data visualisation is particularly designed for multi-temporal and multi-sensor image indexing, where efficient visualisation plays a critical role. The system is applicable to both optical and synthetic aperture radar images. It can index patches and it can also discover temporal patterns in satellite image time series. Three typical case studies are included to show its wide use in EO applications.

Information Content of Very-High-Resolution SAR Images: Semantics, Geospatial Context, and Ontologies

Currently, the amount of collected Earth Observation (EO) data is increasing considerably with a rate of several Terabytes of data per day. As a consequence of this increasing data volume, new concepts for exploration and information retrieval are urgently needed. To this end, we propose to explore satellite image data via an image information mining (IIM) approach in which the main steps are feature extraction, classification, semantic annotation, and interactive query processing. This leads to a new process chain and a robust taxonomy for the retrieved categories capitalizing on human interaction and judgment. We concentrated on land cover categories that can be retrieved from high-resolution synthetic aperture radar (SAR) images of the spaceborne TerraSAR-X instrument, where we annotated different urban areas all over the world and defined a taxonomy element for each prevailing surface cover category. The annotation resulted from a test dataset comprising more than 100 scenes covering diverse areas of Africa, Asia, Europe, the Middle East, and North and South America. The scenes were grouped into several collections with similar source areas and each collection was processed separately in order to discern regional characteristics. In the first processing step, each scene was tiled into patches. Then the features were extracted from each patch by a Gabor filter bank and a support vector machine with relevance feedback classifying the feature sets into user-oriented land cover categories. Finally, the categories were semantically annotated using Google Earth for ground truthing. The annotation followed a multilevel approach that allowed the fusion of information being visible on different resolution levels. The novelty of this paper lies in the fact that a semantic annotation was performed with a large number of high-resolution radar images that allowed the definition of more than 850 surface cover categories. This opens the way toward an automated identification and classification of urban areas, infrastructure (e.g., airports), geographic objects (e.g., mountains), industrial installations, military compounds, vegetation, and agriculture. Applications that may result from this work can be a semantic catalog of urban images to be used in crisis situations or after a disaster. In addition, the proposed taxonomies can become a basis for building a semantic catalog of satellite images. Finally, we defined four powerful types of high-level queries. Querying on such high levels provides new opportunities for users to search an image database for specific parameters or semantic relationships.

Building Virtual Earth Observatories Using Ontologies, Linked Geospatial Data and Knowledge Discovery Algorithms

Advances in remote sensing technologies have allowed us to send an ever-increasing number of satellites in orbit around Earth. As a result, satellite image archives have been constantly increasing in size in the last few years (now reaching petabyte sizes), and have become a valuable source of information for many science and application domains (environment, oceanography, geology, archaeology, security, etc.). TELEIOS is a recent European project that addresses the need for scalable access to petabytes of Earth Observation data and the discovery of knowledge that can be used in applications. To achieve this, TELEIOS builds on scientific databases, linked geospatial data, ontologies and techniques for discovering knowledge from satellite images and auxiliary data sets. In this paper we outline the vision of TELEIOS (now in its second year), and give details of its original contributions on knowledge discovery from satellite images and auxiliary datasets, ontologies, and linked geospatial data.

Data Analytics for Rapid Mapping: Case Study of a Flooding Event in Germany and the Tsunami in Japan Using Very High Resolution SAR Images

In this paper, we present data analytics for a quantitative analysis in a rapid mapping scenario applied for damage assessment of the 2013 floods in Germany and the 2011 tsunami in Japan. These scenarios are created using preand postdisaster TerraSAR-X images and a semi-automated processing chain. All our dataset is tiled into patches and Gabor filters are applied as a primitive feature extraction method to each patch separately. A support vector machine with relevance feedback is implemented in order to group the features into categories. Once all categories are identified, these are semantically annotated using reference data as ground truth. In our investigation, nondamaged and damaged categories were retrieved with their specific taxonomies defined using our previous hierarchical annotation scheme. The classifier supports rapid mapping scenarios (e.g., floods in Germany and tsunami in Japan) and interactive mapping generation. The quantitative damages can be assessed by: 1) flooded agricultural areas (21.66% in the case of floods in Germany and 4.15% in the case of tsunami in Japan) and destroyed aquaculture (2.33% in the case of tsunami in Japan); 2) destroyed transportation infrastructures, such as airport (50% in case tsunami in Japan), bridges, and roads.; and 3) debris that appears in postdisaster images (3.81% in the case of tsunami after the aquaculture was destroyed). The first analysis envisages the floods of Elbe river in June 2013: 30% of the investigated area, about , including agricultural land, forest, river, and some residential and industrial areas close to the river, was covered by water. The second analysis, considering an area of affected by the tsunami, led us to conclude that 3 months after the tsunami, some of the categories returned to previous functionality-the airport, others return to partial functionality such as isolated residents, and some were totally destroyed-the aquaculture. The flooded area was about . The proposed approach goes beyond a simple annotation of the data and provides an intermediate product in order to produce a relevant visual analytics representation of the data. This makes it easier to compare datasets and different quantitative findings in a meaningful manner, accessible both to experts and regular users. Our paper presents an interactive and automatic, fast processing method applicable to large and complex datasets (such as image time series). In addition to enhancing the information content extraction (number of identified categories), this approach enables the discovery and analysis of these categories. The novelty of this paper resides in that this is the first time data analytics have been run on a large dataset and for different scenarios using a semiautomated processing chain.

Land Cover Semantic Annotation Derived from High-Resolution SAR Images

Users of remote sensing images analyzing land cover characteristics are very much interested in classification schemes that define a consistent set of target categories. Up to now, a number of established classification schemes are mainly being used by interpreters of medium-resolution optical satellite images focusing on large-scale land cover. In contrast, we concentrate in this publication on the definition of a new classification scheme for high-resolution synthetic aperture radar (SAR) images that are mostly taken over built-up areas. Here, we can see many small details of buildings, industrial facilities, and infrastructure that have to be classified. However, the appearance of details in high-resolution SAR images is often difficult to understand for human observers, and, therefore, calls for an automated semantic annotation of the target objects that has to follow a number of specific scientific guidelines. We demonstrate that a selection of representative SAR images with subsequent feature extraction and relevance feedback classification during the generation of a classification scheme leads to a reliable definition of a new high-resolution multi-level SAR image classification scheme that can be applied globally for semantic annotation in an automated chain.

Very-High-Resolution SAR Images and Linked Open Data Analytics Based on Ontologies

In this paper, we deal with the integration of multiple sources of information such as Earth observation (EO) synthetic aperture radar (SAR) images and their metadata, semantic descriptors of the image content, as well as other publicly available geospatial data sources expressed as linked open data for posing complex queries in order to support geospatial data analytics. Our approach lays the foundations for the development of richer tools and applications that focus on EO image analytics using ontologies and linked open data. We introduce a system architecture where a common satellite image product is transformed from its initial format into to actionable intelligence information, which includes image descriptors, metadata, image tiles, and semantic labels resulting in an EO-data model. We also create a SAR image ontology based on our EO-data model and a two-level taxonomy classification scheme of the image content. We demonstrate our approach by linking high-resolution TerraSAR-X images with information from CORINE Land Cover (CLC), Urban Atlas (UA), GeoNames, and OpenStreetMap (OSM), which are represented in the standard triple model of the resource description frameworks (RDFs).

How many categories are in very high resolution SAR images

In this paper, we propose to identify the number of categories that can be retrieved from a very high resolution SAR data. The evaluation is done on TerraSAR-X high resolution Spotlight data and the retrieved categories are semi-automatically annotated using as feature vector the Gabor filters; as a classifier the Support Vector Machine, and for ranking the suggested images the relevance feedback. The visualization of the tool was enhanced compared with our previous implementation in order to support the users in his/her approach to search the patches of interest in a large repository. Our dataset consist in 43 scenes that cover as much as possible all the regions over the world. A total of 352 categories are identified that contain urban and non-urban categories.

Study and assessment of selected primitive features behaviour for SAR image description

The main purpose of this study is to define for Synthetic Aperture Radar (SAR) data the primitive feature parameters, the incidence angle, and the orbit direction which can be used further for indexing and querying in the EO systems. The evaluation is done on the high resolution SAR data and the interpretation is realized automatically. In this paper, we propose to study and asses the behavior of the primitive feature extracted methods for images of the same scene with two look angles covering the min-max range of the sensor and with ascending / descending orbit looking. The tests are done on TerraSAR-X products Stripmap and high resolution Spotlight, specially and radiometrically enhanced covering the area of Berlin (Germany) and Ottawa (Canada). To identify the optimal primitive features, incident angle, and orbit direction the Support Vector Machine and as a measure of the classification accuracy the precision/recall were considered.