Michele Volpi

A Survey of Active Learning Algorithms for Supervised Remote Sensing Image Classification

Defining an efficient training set is one of the most delicate phases for the success of remote sensing image classification routines. The complexity of the problem, the limited temporal and financial resources, as well as the high intraclass variance can make an algorithm fail if it is trained with a suboptimal dataset. Active learning aims at building efficient training sets by iteratively improving the model performance through sampling. A user-defined heuristic ranks the unlabeled pixels according to a function of the uncertainty of their class membership and then the user is asked to provide labels for the most uncertain pixels. This paper reviews and tests the main families of active learning algorithms: committee, large margin, and posterior probability-based. For each of them, the most recent advances in the remote sensing community are discussed and some heuristics are detailed and tested. Several challenging remote sensing scenarios are considered, including very high spatial resolution and hyperspectral image classification. Finally, guidelines for choosing the good architecture are provided for new and/or unexperienced user.

Supervised change detection in VHR images using contextual information and support vector machines

In this paper we study an effective solution to deal with supervised change detection in very high geometrical resolution (VHR) images. High within-class variance as well as low between-class variance that characterize this kind of imagery make the detection and classification of ground cover transitions a difficult task. In order to achieve high detection accuracy, we propose the inclusion of spatial and contextual information issued from local textural statistics and mathematical morphology. To perform change detection, two architectures, initially developed for medium resolution images, are adapted for VHR: Direct Multi-date Classification and Difference Image Analysis. To cope with the high intra-class variability, we adopted a nonlinear classifier: the Support Vector Machines (SVM). The proposed approaches are successfully evaluated on two series of pansharpened QuickBird images.

Multi-Modal Change Detection, Application to the Detection of Flooded Areas: Outcome of the 2009–2010 Data Fusion Contest

The 2009-2010 Data Fusion Contest organized by the Data Fusion Technical Committee of the IEEE Geoscience and Remote Sensing Society was focused on the detection of flooded areas using multi-temporal and multi-modal images. Both high spatial resolution optical and synthetic aperture radar data were provided. The goal was not only to identify the best algorithms (in terms of accuracy), but also to investigate the further improvement derived from decision fusion. This paper presents the four awarded algorithms and the conclusions of the contest, investigating both supervised and unsupervised methods and the use of multi-modal data for flood detection. Interestingly, a simple unsupervised change detection method provided similar accuracy as supervised approaches, and a digital elevation model-based predictive method yielded a comparable projected change detection map without using post-event data.

Dense Semantic Labeling of Subdecimeter Resolution Images With Convolutional Neural Networks

Semantic labeling (or pixel-level land-cover classification) in ultrahigh-resolution imagery (<10 cm) requires statistical models able to learn high-level concepts from spatial data, with large appearance variations. Convolutional neural networks (CNNs) achieve this goal by learning discriminatively a hierarchy of representations of increasing abstraction. In this paper, we present a CNN-based system relying on a downsample-then-upsample architecture. Specifically, it first learns a rough spatial map of high-level representations by means of convolutions and then learns to upsample them back to the original resolution by deconvolutions. By doing so, the CNN learns to densely label every pixel at the original resolution of the image. This results in many advantages, including: 1) the state-of-the-art numerical accuracy; 2) the improved geometric accuracy of predictions; and 3) high efficiency at inference time. We test the proposed system on the Vaihingen and Potsdam subdecimeter resolution data sets, involving the semantic labeling of aerial images of 9- and 5-cm resolution, respectively. These data sets are composed by many large and fully annotated tiles, allowing an unbiased evaluation of models making use of spatial information. We do so by comparing two standard CNN architectures with the proposed one: standard patch classification, prediction of local label patches by employing only convolutions, and full patch labeling by employing deconvolutions. All the systems compare favorably or outperform a state-of-the-art baseline relying on superpixels and powerful appearance descriptors. The proposed full patch labeling CNN outperforms these models by a large margin, also showing a very appealing inference time.

Unsupervised Change Detection With Kernels

In this letter, an unsupervised kernel-based approach to change detection is introduced. Nonlinear clustering is utilized to partition in two a selected subset of pixels representing both changed and unchanged areas. Once the optimal clustering is obtained, the learned representatives of each group are exploited to assign all the pixels composing the multitemporal scenes to the two classes of interest. Two approaches based on different assumptions of the difference image are proposed. The first accounts for the difference image in the original space, while the second defines a mapping describing the difference image directly in feature spaces. To optimize the parameters of the kernels, a novel unsupervised cost function is proposed. An evidence of the correctness, stability, and superiority of the proposed solution is provided through the analysis of two challenging change-detection problems.

Semisupervised Manifold Alignment of Multimodal Remote Sensing Images

We introduce a method for manifold alignment of different modalities (or domains) of remote sensing images. The problem is recurrent when a set of multitemporal, multisource, multisensor, and multiangular images is available. In these situations, images should ideally be spatially coregistered, corrected, and compensated for differences in the image domains. Such procedures require massive interaction of the user, involve tuning of many parameters and heuristics, and are usually applied separately. Changes of sensors and acquisition conditions translate into shifts, twists, warps, and foldings of the (typically nonlinear) manifolds where images lie. The proposed semisupervised manifold alignment (SS-MA) method aligns the images working directly on their manifolds and is thus not restricted to images of the same resolutions, either spectral or spatial. SS-MA pulls close together samples of the same class while pushing those of different classes apart. At the same time, it preserves the geometry of each manifold along the transformation. The method builds a linear invertible transformation to a latent space where all images are alike and reduces to solving a generalized eigenproblem of moderate size. We study the performance of SS-MA in toy examples and in real multiangular, multitemporal, and multisource image classification problems. The method performs well for strong deformations and leads to accurate classification for all domains. A MATLAB implementation of the proposed method is provided at http://isp. uv.es/code/ssma.htm.

Automatic Feature Learning for Spatio-Spectral Image Classification With Sparse SVM

Including spatial information is a key step for successful remote sensing image classification. In particular, when dealing with high spatial resolution, if local variability is strongly reduced by spatial filtering, the classification performance results are boosted. In this paper, we consider the triple objective of designing a spatial/spectral classifier, which is compact (uses as few features as possible), discriminative (enhances class separation), and robust (works well in small sample situations). We achieve this triple objective by discovering the relevant features in the (possibly infinite) space of spatial filters by optimizing a margin-maximization criterion. Instead of imposing a filter bank with predefined filter types and parameters, we let the model figure out which set of filters is optimal for class separation. To do so, we randomly generate spatial filter banks and use an active-set criterion to rank the candidate features according to their benefits to margin maximization (and, thus, to generalization) if added to the model. Experiments on multispectral very high spatial resolution (VHR) and hyperspectral VHR data show that the proposed algorithm, which is sparse and linear, finds discriminative features and achieves at least the same performances as models using a large filter bank defined in advance by prior knowledge.

Semantic segmentation of urban scenes by learning local class interactions

Traditionally, land-cover mapping from remote sensing images is performed by classifying each atomic region in the image in isolation and by enforcing simple smoothing priors via random fields models as two independent steps. In this paper, we propose to model the segmentation problem by a discriminatively trained Conditional Random Field (CRF). To this end, we employ Structured Support Vector Machines (SSVM) to learn the weights of an informative set of appearance descriptors jointly with local class interactions. We propose a principled strategy to learn pairwise potentials encoding local class preferences from sparsely annotated ground truth. We show that this approach outperform standard baselines and more expressive CRF models, improving by 4-6 points the average class accuracy on a challenging dataset involving urban high resolution satellite imagery.

Semisupervised Transfer Component Analysis for Domain Adaptation in Remote Sensing Image Classification

In this paper, we study the problem of feature extraction for knowledge transfer between multiple remotely sensed images in the context of land-cover classification. Several factors such as illumination, atmospheric, and ground conditions cause radiometric differences between images of similar scenes acquired on different geographical areas or over the same scene but at different time instants. Accordingly, a change in the probability distributions of the classes is observed. The purpose of this work is to statistically align in the feature space an image of interest that still has to be classified (the target image) to another image whose ground truth is already available (the source image). Following a specifically designed feature extraction step applied to both images, we show that classifiers trained on the source image can successfully predict the classes of the target image despite the shift that has occurred. In this context, we analyze a recently proposed domain adaptation method aiming at reducing the distance between domains, Transfer Component Analysis, and assess the potential of its unsupervised and semisupervised implementations. In particular, with a dedicated study of its key additional objectives, namely the alignment of the projection with the labels and the preservation of the local data structures, we demonstrate the advantages of Semisupervised Transfer Component Analysis. We compare this approach with other both linear and kernel-based feature extraction techniques. Experiments on multi- and hyperspectral acquisitions show remarkable cross- image classification performances for the considered strategy, thus confirming its suitability when applied to remotely sensed images.

Memory-Based Cluster Sampling for Remote Sensing Image Classification

In this paper, we address the problem of semi-automatic definition of training sets for the classification of remotely sensed images. We propose two approaches based on active learning aiming at removing both the proximal (low diversity) and the dense (low exploration during iterations) sampling redundancies. The first is encountered when several samples carrying similar spectral information are selected by the algorithm, while the second occurs when the heuristic is unable to explore undiscovered parts of the feature space during iterations. For this purpose, kernel k-means is used to cluster a set of uncertain candidates in the same space spanned by the kernel function defined in the SVM classification step. Two heuristics are proposed to maximize the speed of convergence to high classification accuracies: The first is based on binary hierarchical partitioning of the set of selected uncertain samples, while the second extends this approach by considering memory in the selection and thus dynamically adapts to the problem throughout the iterations. Experiments on both VHR and hyperspectral imagery confirm fast convergence of the algorithm, that outperforms state-of-the-art sampling schemes.