Lihong Wan

Selective convolutional neural networks and cascade classifiers for remote sensing image classification

ABSTRACT Training convolutional neural network (CNN) architecture fully, using pretrained CNNs as feature extractors, and fine-tuning pretrained CNNs on target datasets are three popular strategies used in state-of-the-art methods for remote sensing image classification. The full training strategy requires large-scale training dataset, whereas the fine-tuning strategy requires a pretrained model to resume network learning. In this study, we propose a new strategy based on selective CNNs and cascade classifiers to improve the classification accuracy of remote sensing images relative to single CNN. First, we conduct a comparative study of existing pretrained CNNs in terms of data augmentation and the use of fully connected layers. Second, selective CNNs, which based on class separability criterion, are presented to obtain an optimal combination from multiple pretrained models. Finally, classification accuracy is improved by introducing two-stage cascade linear classifiers, the prediction probability of which in the first stage is used as input for the second stage. Experiments on three public remote sensing datasets demonstrate the effectiveness of the proposed method in comparison with state-of-the-art methods.

Affine Invariant Description and Large-Margin Dimensionality Reduction for Target Detection in Optical Remote Sensing Images

A novel target detection method based on affine invariant interest point detection, feature encoding, and large-margin dimensionality reduction (LDR) is proposed for optical remote sensing images. First, four types of interest point detectors are introduced, and their performance in extracting low-level affine invariant descriptors using affine shape estimation is compared. Such a description can deal with significant affine transformations, including viewpoints. Second, feature encoding, which extends bag-of-words (BOW) by encoding high-order statistics, is selected to generate mid-level representation. Finally, LDR based on the large-margin constraint and stochastic subgradient is introduced to make the high-dimensional mid-level representation applicable for target detection. The experiments on aircraft and vehicle detections illustrate the effectiveness of the affine invariant description and LDR (compared with principal component analysis) in improving the detection performance. The experiments also demonstrate the effectiveness of the proposed method compared with popular approaches including Gabor, HOG, LBP, BOW, and R-CNN.

Improving the Separability of Deep Features with Discriminative Convolution Filters for RSI Classification

The extraction of activation vectors (or deep features) from the fully connected layers of a convolutional neural network (CNN) model is widely used for remote sensing image (RSI) representation. In this study, we propose to learn discriminative convolution filter (DCF) based on class-specific separability criteria for linear transformation of deep features. In particular, two types of pretrained CNN called CaffeNet and VGG-VD16 are introduced to illustrate the generality of the proposed DCF. The activation vectors extracted from the fully connected layers of a CNN are rearranged into the form of an image matrix, from which a spatial arrangement of local patches is extracted using sliding window strategy. DCF learning is then performed on each local patch individually to obtain the corresponding discriminative convolution kernel through generalized eigenvalue decomposition. The proposed DCF learning characterizes that a convolutional kernel with small size (e.g., 3 × 3 pixels) can be effectively learned on a small-size local patch (e.g., 8 × 8 pixels), thereby ensuring that the linear transformation of deep features can maintain low computational complexity. Experiments on two RSI datasets demonstrate the effectiveness of DCF in improving the classification performances of deep features without increasing dimensionality.

A noise removal approach for object-based classification of VHR imagery via post-classification

The pixel-based classification of remotely sensed images always produces a large amount of “speckled” or “salt and pepper” noises. Both post-classification smoothing and object-based classification techniques have been proposed to tackle this problem. However, most of them are not adequate to deal with the noises in object-based classification of very high resolution (VHR) remote sensing imagery, because a lot of noisy regions will be produced by image segmentation and the existing post-classification approaches generally are tailored towards pixel-based classification. This paper proposes a novel noise removal approach for object-based classification of VHR imagery via post-classification. It includes four phases: firstly, an image is segmented into homogeneous regions; secondly, all regions are classified according to their spectral and texture features; thirdly, noisy regions are distinguished by using shape features. Finally, the noisy regions are removed by using contextual features. Experimental results show the proposed approach is effective and can improve the overall accuracy of classification of VHR remote sensing imagery.

Non-convex joint bilateral guided depth upsampling

Blurring depth edges and texture copy artifacts are challenging issues for guided depth map upsampling. They are caused by the inconsistency between depth edges and corresponding color edges. In this paper, we extend the well-known Joint Bilateral Upsampling (JBU) (Kopf et al. 2007) with a novel non-convex optimization framework for guided depth map upsampling, which is denoted as Non-Convex JBU (NCJBU). We show that the proposed NCJBU can well handle the edge inconsistency by making use of the property of both the guidance color image and the depth map. Through comprehensive experiments, we show that our NCJBU can preserve sharp depth edges and properly suppress texture copy artifacts. In addition, we present a data driven scheme to properly determine the parameter in our model such that fine details and sharp depth edges are well preserved even for a large upsampling factor (e.g., 8 ×). Experimental results on both simulated and real data show the effectiveness of our method.

Local feature representation based on linear filtering with feature pooling and divisive normalization for remote sensing image classification

Abstract. We propose a local feature representation based on two types of linear filtering, feature pooling, and nonlinear divisive normalization for remote sensing image classification. First, images are decomposed using a bank of log-Gabor and Gaussian derivative filters to obtain filtering responses that are robust to changes in various lighting conditions. Second, the filtering responses computed using the same filter at nearby locations are pooled together to enhance position invariance and compact representation. Third, divisive normalization with channel-wise strategy, in which each pooled feature is divided by a common factor plus the sum of the neighboring features to reduce dependencies among nearby locations, is introduced to extract divisive normalization features (DNFs). Power-law transformation and principal component analysis are applied to make DNF significantly distinguishable, followed by feature fusion to enhance local description. Finally, feature encoding is used to aggregate DNFs into a global representation. Experiments on 21-class land use and 19-class satellite scene datasets demonstrate the effectiveness of the channel-wise divisive normalization compared with standard normalization across channels and the fusion of the two types of linear filtering in improving classification accuracy. The experiments also illustrate that the proposed method is competitive with state-of-the-art approaches.

Fusing local texture description of saliency map and enhanced global statistics for ship scene detection

In this paper, we introduce a new feature representation based on fusing local texture description of saliency map and enhanced global statistics for ship scene detection in very high-resolution remote sensing images in inland, coastal, and oceanic regions. First, two low computational complexity methods are adopted. Specifically, the Itti attention model is used to extract saliency map, from which local texture histograms are extracted by LBP with uniform pattern. Meanwhile, Gabor filters with multi-scale and multi-orientation are convolved with the input image to extract Gist, means and variances which are used to form the enhanced global statistics. Second, sliding window-based detection is applied to obtain local image patches and extract the fusion of local and global features. SVM with RBF kernel is then used for training and classification. Such detection manner could remove coastal and oceanic regions effectively. Moreover, the ship scene region of interest can be detected accurately. Experiments on 20 very high-resolution remote sensing images collected by Google Earth shows that the fusion feature has advantages than LBP, Saliency map-based LBP and Gist, respectively. Furthermore, desirable results can be obtained in the ship scene detection.

A Comparative Study for Contour Detection Using Deep Convolutional Neural Networks

Contour detection plays an important role in a wide range of applications such as image segmentation, object detection, shape matching, scene understanding, etc. In this study, we conduct a comprehensive analysis of contour detection using existing convolutional neural network (CNN) architectures. Given that contour detection can be considered as a classification task (e.g., contour or non-contour), six types of pretrained CNN (trained on ImageNet dataset) are individually used for domain-specific fine-tuning on contour dataset. The contour detection can then be achieved by sliding-window strategy, in which each image window (corresponding to a local patch) is used to extract features followed by classification. The features extraction is implemented by extracting the activation vectors from the fully-connected layers (except for the classification layer) of a fine-tuned CNN. Random forest classifier is adopted to predict whether the central pixel of a local patch is passed by contour or not. Experiments on a widely-used dataset called Berkeley Segmentation Data Set (BSDS500) demonstrate that fine-tuning technique can significantly improve the performance of contour detection.

Face Recognition with Convolutional Neural Networks and subspace learning

Deep learning is widely used in computer vision. In this study, we present a new method based on Convolutional Neural Networks (CNN) and subspace learning for face recognition under two circumstances. A very deep CNN architecture called VGG-Face, which learned on a large scale database, is used as feature extractor to extract the activation vector of the fully connected layer in the CNN architecture. Then, two types of subspace learning methods, namely, linear discriminate analysis (LDA) and whitening principal component analysis (WPCA), are respectively introduced to learn the subspace of the activation vectors for face recognition under multiple samples per subject and single sample per subject circumstances. The goals of applying subspace learning to the activation vectors are obtaining compact representation (dimensionality reduction) and performance improvement. Experiments on two face databases (CMU PIE and FERET) demonstrate the effectiveness of VGG-Face + LDA and VGG-Face + WPCA, compared with state-of-the-art methods.

Age and gender classification in the wild with unsupervised feature learning

Abstract. Inspired by unsupervised feature learning (UFL) within the self-taught learning framework, we propose a method based on UFL, convolution representation, and part-based dimensionality reduction to handle facial age and gender classification, which are two challenging problems under unconstrained circumstances. First, UFL is introduced to learn selective receptive fields (filters) automatically by applying whitening transformation and spherical k-means on random patches collected from unlabeled data. The learning process is fast and has no hyperparameters to tune. Then, the input image is convolved with these filters to obtain filtering responses on which local contrast normalization is applied. Average pooling and feature concatenation are then used to form global face representation. Finally, linear discriminant analysis with part-based strategy is presented to reduce the dimensions of the global representation and to improve classification performances further. Experiments on three challenging databases, namely, Labeled faces in the wild, Gallagher group photos, and Adience, demonstrate the effectiveness of the proposed method relative to that of state-of-the-art approaches.