Marin Ferecatu

Interactive Remote-Sensing Image Retrieval Using Active Relevance Feedback

As the resolution of remote-sensing imagery increases, the full complexity of the scenes becomes increasingly difficult to approach. User-defined classes in large image databases are often composed of several groups of images and span very different scales in the space of low-level visual descriptors. The interactive retrieval of such image classes is then very difficult. To address this challenge, we evaluate here, in the context of satellite image retrieval, two general improvements for relevance feedback using support vector machines (SVMs). First, to optimize the transfer of information between the user and the system, we focus on the criterion employed by the system for selecting the images presented to the user at every feedback round. We put forward an active-learning selection criterion that minimizes redundancy between the candidate images shown to the user. Second, for image classes spanning very different scales in the low-level description space, we find that a high sensitivity of the SVM to the scale of the data brings about a low retrieval performance. We argue that the insensitivity to scale is desirable in this context, and we show how to obtain it by the use of specific kernel functions. Experimental evaluation of both ranking and classification performance on a ground-truth database of satellite images confirms the effectiveness of our approach

Semantic interactive image retrieval combining visual and conceptual content description

We address the challenge of semantic gap reduction for image retrieval through an improved support vector machines (SVM)-based active relevance feedback framework, together with a hybrid visual and conceptual content representation and retrieval. We introduce a new feature vector based on projecting the keywords associated to an image on a set of “key concepts” with the help of an external lexical database. We then put forward two improvements of SVM-based relevance feedback method. First, to optimize the transfer of information between the user and the system, we introduce a new active learning selection criterion that minimizes redundancy between the candidate images shown to the user. Second, as most image classes span a wide range of scales in the description space, we argue that the insensitivity of the SVM to the scale of the data is desirable in this context and we show how to obtain it by using specific kernel functions. Experimental evaluations show that the joint use of the new concept-based feature vector and the visual features with our relevance feedback scheme can significantly improve the quality of the results.

Processing of Extremely High-Resolution LiDAR and RGB Data: Outcome of the 2015 IEEE GRSS Data Fusion Contest - Part A: 2-D Contest

In this paper, we discuss the scientific outcomes of the 2015 data fusion contest organized by the Image Analysis and Data Fusion Technical Committee (IADF TC) of the IEEE Geoscience and Remote Sensing Society (IEEE GRSS). As for previous years, the IADF TC organized a data fusion contest aiming at fostering new ideas and solutions for multisource studies. The 2015 edition of the contest proposed a multiresolution and multisensorial challenge involving extremely high-resolution RGB images and a three-dimensional (3-D) LiDAR point cloud. The competition was framed in two parallel tracks, considering 2-D and 3-D products, respectively. In this paper, we discuss the scientific results obtained by the winners of the 2-D contest, which studied either the complementarity of RGB and LiDAR with deep neural networks (winning team) or provided a comprehensive benchmarking evaluation of new classification strategies for extremely high-resolution multimodal data (runner-up team). The data and the previously undisclosed ground truth will remain available for the community and can be obtained at http://www.grss-ieee.org/community/technical-committees/data-fusion/2015-ieee-grss-data-fusion-contest/. The 3-D part of the contest is discussed in the Part-B paper [1].

Retrieval of difficult image classes using svd-based relevance feedback

User-defined classes in large generalist image databases are often composed of several groups of images and span very different scales in the space of low-level visual descriptors. The interactive retrieval of such image classes is then very difficult. To addess his challenge, we propose and evaluate here two general mprovements of SVM-based relevance feedback methods. First, to optimize the transfer of information between the user and the system, we focus on the criterion employed by the system for selecting the images presented to the user at every feedback round. We put forward a new active learning selection criterion that minimizes redundancy between the candidate images shown to the user. Second, for image classes having very different scales, we find that a high sensitivity of the SVM to the scale of the data brings about a low retrieval performance. We then argue that insensitivity to scale is desirable in this context and we show how to obtain it by the use of specific kernel functions. The experimental evaluation of both ranking and classification performance on several image databases confirms the effectiveness of our selection criterion and of the use of kernels that reduce the sensitivity of SVMs to the scale of the data

Benchmarking classification of earth-observation data: From learning explicit features to convolutional networks

In this paper, we address the task of semantic labeling of multisource earth-observation (EO) data. Precisely, we benchmark several concurrent methods of the last 15 years, from expert classifiers, spectral support-vector classification and high-level features to deep neural networks. We establish that (1) combining multisensor features is essential for retrieving some specific classes, (2) in the image domain, deep convolutional networks obtain significantly better overall performances and (3) transfer of learning from large generic-purpose image sets is highly effective to build EO data classifiers.

Pattern Retrieval in Large Image Databases Using Multiscale Coarse-to-Fine Cascaded Active Learning

Pattern retrieval is a fundamental challenge in machine learning but is often subject to the problem of gathering enough labeled examples of the target pattern, and also to the computational complexity inherent to the training and the evaluation of complex classifier functions on large databases. In this paper, we propose a hierarchical top-down processing scheme for pattern retrieval in high-volume high-resolution optical satellite image repositories. We learn via a multistage active learning process a cascade of classifiers working each at a certain scale on a patch-based representation of images. At each stage of the hierarchy, we seek to eliminate large parts of images considered as nonrelevant, the purpose being to set the focus at the finest scales on more promising and as spatially limited as possible areas. Our scheme is based on the fact that by reducing the size of the analysis window (i.e., the size of the patch), we better capture the properties of the targeted object. The cascaded hierarchy is introduced to compensate for the extra computational burden incurred by diminishing the size of the patch, which causes an explosion of the number of patches to process. Unlike most other retrieval methods, which require large training sets and costly offline training, we propose a cascaded active learning strategy to build a classifier at each level of the hierarchy, and we provide a new Multiple Instance Learning algorithm to propagate automatically the training examples from one level of the hierarchy to the other. Two study cases are performed for validation. The first is a test on a database of 61-cm resolution QuickBird panchromatic images and the second is an example of temporal pattern retrieval from a database of Synthetic Aperture Radar (SAR) image time series. These tests show that our method achieves a reduction in the number of computations of two orders of magnitude, while keeping the same accuracy level as recent state-of-the-art methods.

A comparative study of diversity methods for hybrid text and image retrieval approaches

This article compares eight different diversity methods: 3 based on visual information, 1 based on date information, 3 adapted to each topic based on location and visual information; finally, for completeness, 1 based on random permutation. To compare the effectiveness of these methods, we apply them on 26 runs obtained with varied methods from different research teams and based on different modalities. We then discuss the results of the more than 200 obtained runs. The results show that query-adapted methods are more effcient than nonadapted method, that visual only runs are more difficult to diversify than text only and text-image runs, and finally that only few methods maximize both the precision and the cluster recall at 20 documents.

Mining large satellite image repositories using semi-supervised methods

The increasing number and resolution of earth observation (EO) imaging sensors has had a significant impact on both the acquired image data volume and the information content in images. There is consequently a strong need for highly efficient search tools for EO image databases and for search methods to automatically identify and recognize structures within EO images. In this paper, we present a concept for an earth observation image data mining system mixing an auto-annotation component with a category search engine which combines a generic image class search and an object detection feature. The proposed concept relies thus on three distinct components which are detailed successively: in the first part, we describe the auto-annotation component, in the second part, the generic category search engine and in the third part, the object detection tool. In the concluding part of the paper, we provide an insight into how these three components can be related to each other and used in a complementary way to arrive at a system which combines the advantages of both the auto-annotation systems and the category search engines.

Cascaded active learning for object retrieval using multiscale coarse to fine analysis

In this paper, we describe an active learning scheme which performs coarse to fine testing using a multiscale patch-based representation of images to retrieve objects in large satellite image repositories. The proposed hierarchical top-down approach reduces step by step the size of the analysis window, eliminating each time large parts of the images considered as non-relevant. Unlike most object detection methods which requires large training sets and costly offline training, we use an active learning strategy to build a classifier at each level of the hierarchy and we propose an algorithm to propagate automatically the training examples from one level to the other.

Sample selection strategies for relevance feedback in region-based image retrieval

The success of the relevance feedback search paradigm in image retrieval is influenced by the selection strategy employed by the system to choose the images presented to the user for providing feedback. Indeed, this strategy has a strong effect on the transfer of information between the user and the system. Using SVMs, we put forward a new active learning selection strategy that minimizes redundancy between the examples. We focus on region-based image retrieval and we expect our approach to produce better results than existing selection strategies. Experimental evidence in the context of generalist image databases confirms the efectiveness of this selection strategy.