Peng Ren

Deep learning in remote sensing scene classification: a data augmentation enhanced convolutional neural network framework

The recent emergence of deep learning for characterizing complex patterns in remote sensing imagery reveals its high potential to address some classic challenges in this domain, e.g. scene classification. Typical deep learning models require extremely large datasets with rich contents to train a multilayer structure in order to capture the essential features of scenes. Compared with the benchmark datasets used in popular deep learning frameworks, however, the volumes of available remote sensing datasets are particularly limited, which have restricted deep learning methods from achieving full performance gains. In order to address this fundamental problem, this article introduces a methodology to not only enhance the volume and completeness of training data for any remote sensing datasets, but also exploit the enhanced datasets to train a deep convolutional neural network that achieves state-of-the-art scene classification performance. Specifically, we propose to enhance any original dataset by applying three operations – flip, translation, and rotation to generate augmented data – and use the augmented dataset to train and obtain a more descriptive deep model. The proposed methodology is validated in three recently released remote sensing datasets, and confirmed as an effective technique that significantly contributes to potentially revolutionary changes in remote sensing scene classification, empowered by deep learning.

Target Tracking Based on Biological-Like Vision Identity via Improved Sparse Representation and Particle Filtering

To effectively track targets under partial occlusion and illumination variation, an improved target tracking method based on combination of sparse representation and particle filtering is proposed in this paper. We regard the candidate target particle set as redundant dictionary and the target template as observation signal to reduce the computational complexity and enhance the real-time performance of target tracking. Besides, to enhance tracking robustness for better adaption to illumination and occlusion, the density histogram, local binary pattern feature fusion, trivial templates and energy control parameters are also utilized in this study. Finally, extensive simulation experiments under different circumstances show that the proposed method performs better compared with other methods, and the average computation time decreases greatly.

Clustering-Oriented Multiple Convolutional Neural Networks for Single Image Super-Resolution

In contrast to the human visual system (HVS) that applies different processing schemes to visual information of different textural categories, most existing deep learning models for image super-resolution tend to exploit an indiscriminate scheme for processing one whole image. Inspired by the human cognitive mechanism, we propose a multiple convolutional neural network framework trained based on different textural clusters of image local patches. To this end, we commence by grouping patches into K clusters via K-means, which enables each cluster center to encode image priors of a certain texture category. We then train K convolutional neural networks for super-resolution based on the K clusters of patches separately, such that the multiple convolutional neural networks comprehensively capture the patch textural variability. Furthermore, each convolutional neural network characterizes one specific texture category and is used for restoring patches belonging to the cluster. In this way, the texture variation within a whole image is characterized by assigning local patches to their closest cluster centers, and the super-resolution of each local patch is conducted via the convolutional neural network trained by its cluster. Our proposed framework not only exploits the deep learning capability of convolutional neural networks but also adapts them to depict texture diversities for super-resolution. Experimental super-resolution evaluations on benchmark image datasets validate that our framework achieves state-of-the-art performance in terms of peak signal-to-noise ratio and structural similarity. Our multiple convolutional neural network framework provides an enhanced image super-resolution strategy over existing single-mode deep learning models.

Game theoretic hypergraph matching for multi-source image correspondences

Game theoretic hypergraph matching framework for multi-source image correspondences.The game evolution guarantees the optimal matching results.A coarse to fine strategy for establishing the game association hypergraph.Game association hypergraphs of reasonably small sizes.Small hypergraphs possibly reduce the computational complexity for matching. In this paper, we develop a game theoretic hypergraph matching framework which enables refining feature correspondences of multi-source images. For images obtained from different sources, we commence by characterizing game strategies in terms of coarse correspondences based on feature similarities. In this scenario, mismatches tend to occur because one feature descriptor may exhibit certain ambiguity in characterizing feature similarities for multi-source images. To address this shortcoming, we develop payoffs for independent game players in terms of higher order strategy similarities, and identify strategies which extinguish in an evolutionary game for achieving maximum average payoff. To render a convenient computational framework, we establish an association uniform hypergraph, referred to as the game hypergraph, with each vertex representing a strategy and each uniform hyperedge characterizing a payoff for independent players. We then exploit the BaumEagon scheme for computing the maximum average game payoff for Nash equilibrium within the association hypergraph. Our method is invariant to scene scale variation because of the higher order structures characterized by the strategy similarities. Furthermore, our method is computationally more efficient than existing hypergraph matching methods because of coarse correspondences. Experimental results show the effectiveness of our method for refining multi-source feature correspondences.

Dual Smoothing for Marine Oil Spill Segmentation

We present a novel marine oil spill segmentation method that characterizes two smoothing modules at the label level and the pixel level separately. At the label level, we exploit the rolling guidance filter for smoothing the label cost volumes. It enables scale-aware labeling and thus alleviates the ambiguous segmentation that blurs the detailed structures of oil spills. At the pixel level, we adapt a cooperative model for smoothing higher order pixel variations, which has the potential of preserving elongated strips that often arise in oil spills. We integrate the two smoothing modules operating at different levels into an energy minimization formulation, which is referred to as dual smoothing. The coupling of the two smoothing modules enables an effective complement to each other such that the specific structures of oil spills are accurately characterized. We compute the optimal labeling of the dual-smoothing framework based on graph cuts. The proposed dual-smoothing framework is especially effective in segmenting elongated and detailed oil spills, and the experimental results demonstrate its advantages over thresholding- and graph-cut-based segmentations.

A Sensor Self-aware Distributed Consensus Filter for Simultaneous Localization and Tracking

Background/IntroductionSimultaneous localization and tracking (SLAT) has become a very hot topic in both academia and industry for its potential wide applications in robotic equipment, sensor networks and smart devices. In order to exploit the advantages supported by state filtering and parameter estimation, researchers have proposed adaptive structures for solving SLAT problems. Existing solutions for SLAT problems that rely on belief propagation often have limited accuracy or high complexity. To adapt the brain decision mechanism for solving SLAT problems, we introduce a specific framework that is suitable for wireless sensor networks.Methods Motivated by the high efficiency and performance of brain decision making built upon partial information and information updating, we propose a cognitively distributed SLAT algorithm based on an adaptive distributed filter, which is composed of two stages for target tracking and sensor localization. The first stage is consensus filtering that updates the target state with respect to each sensor. The second stage employs a recursive parameter estimation that exploits an on-line optimization method for refining the sensor localization. As an integrated framework, each consensus filter is specific to a separate sensor subsystem and gets feedback information from its parameter estimation.ResultsThe performance comparison in terms of positioning accuracy with respect to RMSE is shown and the simulation results demonstrate that the proposed ICF-RML performs better than the BPF-RML. This is expected since the distributed estimation with sufficient communication mechanism often achieves higher accuracy than that of less sufficient cases. Furthermore, the performance of the ICF-RML is comparable with that of the BPF-RML even if the latter assumes known prior network topology. We also observe from the results of tracking errors that ICF-RML accomplishes a remarkable improvement in the precision of target tracking and achieves more stable convergence than BPF-RML, in the scenario that all sensors are used to calculate the effect from data association errors.ConclusionWe apply this approach to formulate the SLAT problem and propose an effective solution, summarized in the paper. For small-size sensor networks with Gaussian distribution, our algorithm can be implemented through a distributed version of weighted information filter and a consensus protocol. Comparing the existing method, our solution shows a higher accuracy in estimation but with less complexity.

Augmented filtering based on information weighted consensus fusion for simultaneous localization and tracking via wireless sensor networks

This paper develops a novel augmented filtering framework based on information weighted consensus fusion, to achieve the simultaneous localization and tracking (SLAT) via wireless sensor networks (WSNs). By integrating augmented transition and observation models, we formulate a dynamical system that encodes both the target moving manners and coarse sensor locations in an augmented state. We then conduct augmented filtering based on augmented extended Kalman filters to estimate the augmented state. We further refine our target estimate according to information weighted consensus filtering which fuses the target information obtained from neighboring sensors. The fused information is fed back as the target estimate to the augmented filter. Our framework is computationally efficient because it only requires neighboring sensor communications. Experiments on SLAT problem validate the effectiveness of the proposed algorithm in terms of tracking accuracy and localization precision in limited ranging conditions.

Deep learning for vision-based micro aerial vehicle autonomous landing:

Vision-based techniques are widely used in micro aerial vehicle autonomous landing systems. Existing vision-based autonomous landing schemes tend to detect specific landing landmarks by identifying their straightforward visual features such as shapes and colors. Though efficient to compute, these schemes only apply to landmarks with limited variability and require strict environmental conditions such as consistent lighting. To overcome these limitations, we propose an end-to-end landmark detection system based on a deep convolutional neural network, which not only easily scales up to a larger number of various landmarks but also exhibit robustness to different lighting conditions. Furthermore, we propose a separative implementation strategy which conducts convolutional neural network training and detection on different hardware platforms separately, i.e. a graphics processing unit work station and a micro aerial vehicle on-board system, subject to their specific implementation requirements. To evaluate the performance of our framework, we test it on synthesized scenarios and real-world videos captured by a quadrotor on-board camera. Experimental results validate that the proposed vision-based autonomous landing system is robust to landmark variability in different backgrounds and lighting situations.

Characterization of Received Signal Strength Perturbations Using Allan Variance

The received signal strength (RSS) of wireless signals conveys important information that has been widely used in wireless communications, localization, and tracking. Traditional RSS-based research and applications model the RSS signal using a deterministic component plus a white noise term. This paper investigates the assumption of white noise to have a further understanding of the RSS signal and proposes a methodology based on the Allan variance (AVAR) to characterize it. Using AVAR, we model the RSS unknown perturbations as correlated random terms. These terms can account for both colored noise or other effects such as shadowing or small-scale fading. Our results confirm that AVAR can be used to obtain a flexible model of the RSS perturbations, as expressed by colored noise components . The study is complemented by introducing two straightforward applications of the proposed methodology: the modeling and simulation of RSS noise using Wiener processes, and RSS localization using the extended Kalman filter.

Real-Time Top-View People Counting Based on a Kinect and NVIDIA Jetson TK1 Integrated Platform

In this paper, we describe how to establish an embedded framework for real-time top-view people counting. The development of our system consists of two parts, i.e. establishing an embedded signal processing platform and designing a people counting algorithm for the embedded system. For the hardware platform construction, we use Kinect as the camera and exploit NVIDIA Jetson TK1 board as the embedded processing platform. We describe how to build a channel to make Kinect for windows version 2.0 communicate with Jetson TK1. Based on the embedded system, we adapt a water filling based scheme for top-view people counting, which integrates head detection based on water drop, people tracking and counting. Gaussian Mixture Model is used to construct and update the background model. The moving people in each video frame are extracted using background subtraction method. Additionally, the water filling algorithm is used to segment head area as Region Of Interest(ROI). Tracking and counting people are performed by calculating the distance of ROI center point before and after the frame. The whole framework is flexible and practical for real-time application.