Pierre Blanchart

A Semi-Supervised Algorithm for Auto-Annotation and Unknown Structures Discovery in Satellite Image Databases

The increasing number and resolution of earth observation (EO) imaging sensors has had a significant impact on both the acquired image data volume and the information content in images. There is consequently a strong need for highly efficient search tools for EO image databases and for search methods to automatically identify and recognize structures within EO images. Content Based Image Retrieval (CBIR) and automatic image annotation systems have been designed to tackle the problem of image retrieval in large image databases. These two systems achieve a common goal, that is to learn the mapping function between low-level visual features and high-level image semantics. A setup, which has hardly been explored in annotating systems and which is the rule rather than the exception, is the case when the training database used to learn the mapping function is not exhaustive regarding semantic classes present in the images. This means that there exists unknown image classes for which there is no training examples in the training database. In this paper, we propose a semi-supervised method for auto-annotating satellite image databases and discovering unknown semantic image classes in these databases. The idea is to incorporate into the learning process the unannotated data which by definition contain the unknown image classes. The latter are considered to be latent structures in the data that appear when we train a hierarchical latent variable model with both the labeled and unlabeled data. We also show that, in our case, the use of unlabeled data leads to more reliable estimates regarding the model parameters. We present experimental results on a synthetic dataset, making a comparison of our algorithm with a semi-supervised Support Vector Machine (S3VM) on this dataset. We also demonstrate the effectiveness of our unknown image classes discovery procedure on a database of SPOT5 satellite images. We show that the results obtained on this database are rather positive since the new structures detected correspond to semantic classes which are not represented in the training database.

Pattern Retrieval in Large Image Databases Using Multiscale Coarse-to-Fine Cascaded Active Learning

Pattern retrieval is a fundamental challenge in machine learning but is often subject to the problem of gathering enough labeled examples of the target pattern, and also to the computational complexity inherent to the training and the evaluation of complex classifier functions on large databases. In this paper, we propose a hierarchical top-down processing scheme for pattern retrieval in high-volume high-resolution optical satellite image repositories. We learn via a multistage active learning process a cascade of classifiers working each at a certain scale on a patch-based representation of images. At each stage of the hierarchy, we seek to eliminate large parts of images considered as nonrelevant, the purpose being to set the focus at the finest scales on more promising and as spatially limited as possible areas. Our scheme is based on the fact that by reducing the size of the analysis window (i.e., the size of the patch), we better capture the properties of the targeted object. The cascaded hierarchy is introduced to compensate for the extra computational burden incurred by diminishing the size of the patch, which causes an explosion of the number of patches to process. Unlike most other retrieval methods, which require large training sets and costly offline training, we propose a cascaded active learning strategy to build a classifier at each level of the hierarchy, and we provide a new Multiple Instance Learning algorithm to propagate automatically the training examples from one level of the hierarchy to the other. Two study cases are performed for validation. The first is a test on a database of 61-cm resolution QuickBird panchromatic images and the second is an example of temporal pattern retrieval from a database of Synthetic Aperture Radar (SAR) image time series. These tests show that our method achieves a reduction in the number of computations of two orders of magnitude, while keeping the same accuracy level as recent state-of-the-art methods.

Mining large satellite image repositories using semi-supervised methods

The increasing number and resolution of earth observation (EO) imaging sensors has had a significant impact on both the acquired image data volume and the information content in images. There is consequently a strong need for highly efficient search tools for EO image databases and for search methods to automatically identify and recognize structures within EO images. In this paper, we present a concept for an earth observation image data mining system mixing an auto-annotation component with a category search engine which combines a generic image class search and an object detection feature. The proposed concept relies thus on three distinct components which are detailed successively: in the first part, we describe the auto-annotation component, in the second part, the generic category search engine and in the third part, the object detection tool. In the concluding part of the paper, we provide an insight into how these three components can be related to each other and used in a complementary way to arrive at a system which combines the advantages of both the auto-annotation systems and the category search engines.

Cascaded active learning for object retrieval using multiscale coarse to fine analysis

In this paper, we describe an active learning scheme which performs coarse to fine testing using a multiscale patch-based representation of images to retrieve objects in large satellite image repositories. The proposed hierarchical top-down approach reduces step by step the size of the analysis window, eliminating each time large parts of the images considered as non-relevant. Unlike most object detection methods which requires large training sets and costly offline training, we use an active learning strategy to build a classifier at each level of the hierarchy and we propose an algorithm to propagate automatically the training examples from one level to the other.

Cascade active learning for SAR image annotation

In this paper, a novel active learning approach and system incorporating multiple instance learning for SAR image mining and annotation is introduced. Based on a multiscale and hierarchial patch based image representation, a cascade classifier is learned at different levels. At each level of the hierarchy, a SVM classifier is trained based on active learning and the training sample propagation between different levels is achieved through Multiple Instance SVM (MI-SVM). Classification at the higher level is applied only to the positive patches obtained at the previous level, which can significantly reduce the burden of computation in the case of large data set. Performance has been evaluated through a large data set, which shows promising gain not only in accuracy but also in computation.

Semi-supervised learning and discovery of unkown structures among data: Application to satellite image annotation

In this paper, we present a semi-supervised method for auto-annotating image collections and discovering unknown structures among them. The approach relies on the existence of only a small training database of annotated examples. First, a fully-supervised algorithm using annotated samples is presented. Next, we introduce a semi-supervised procedure which allows us to incorporate unannotated samples and to infer the existence of unknown structures, that is, the existence of new image classes which are not represented in the training database. Finally, we present experimental results from a database of satellite images and briefly mention the possibility of reusing the presented approach as a basis for more complex systems such as Content Based Image Retrieval (CBIR) systems.

Local integrity constraints for structure detection and segmentation in high-resolution earth observation images

Considering the idea that objects in images have a higher local structural integrity than the background they lie into, we propose a method that learns a supervised distance characterizing the membership of a pair of elements to the target structure. We test our ideas by applying them to the task of extracting semantic structures in high resolution Earth Observation images. The results show that the method works well in many situations when there is no training dataset. The limits of the method are also discussed.

Active learning using the data distribution for interactive image classification and retrieval

In the context of image search and classification, we describe an active learning strategy that relies on the intrinsic data distribution modeled as a mixture of Gaussians to speed up the learning of the target class using an interactive relevance feedback process. The contributions of our work are twofold: First, we introduce a new form of a semi-supervised C-SVM algorithm that exploits the intrinsic data distribution by working directly on equiprobable envelopes of Gaussian mixture components. Second, we introduce an active learning strategy which allows to interactively adjust the equiprobable envelopes in a small number of feedback steps. The proposed method allows the exploitation of the information contained in the unlabeled data and does not suffer from the drawbacks inherent to semi-supervised methods, e.g. computation time and memory requirements. Tests performed on a database of high-resolution satellite images and on a database of color images show that our system compares favorably, in terms of learning speed and ability to manage large volumes of data, to the classic approach using SVM active learning.