Qilin Zhang

Iterative Sparse Asymptotic Minimum Variance Based Approaches for Array Processing

This paper presents a series of user parameter-free iterative Sparse Asymptotic Minimum Variance (SAMV) approaches for array processing applications based on the asymptotically minimum variance (AMV) criterion. With the assumption of abundant snapshots in the direction-of-arrival (DOA) estimation problem, the signal powers and noise variance are jointly estimated by the proposed iterative AMV approach, which is later proved to coincide with the Maximum Likelihood (ML) estimator. We then propose a series of power-based iterative SAMV approaches, which are robust against insufficient snapshots, coherent sources and arbitrary array geometries. Moreover, to overcome the direction grid limitation on the estimation accuracy, the SAMV-Stochastic ML (SAMV-SML) approaches are derived by explicitly minimizing a closed form stochastic ML cost function with respect to one scalar paramter, eliminating the need of any additional grid refinement techniques. To assist the performance evaluation, approximate solutions to the SAMV approaches are also provided for high signal-to-noise ratio (SNR) and low SNR scenarios. Finally, numerical examples are generated to compare the performances of the proposed approaches with those of the existing ones.

Can Visual Recognition Benefit from Auxiliary Information in Training

We examine an under-explored visual recognition problem, where we have a main view along with an auxiliary view of visual information present in the training data, but merely the main view is available in the test data. To effectively leverage the auxiliary view to train a stronger classifier, we propose a collaborative auxiliary learning framework based on a new discriminative canonical correlation analysis. This framework reveals a common semantic space shared across both views through enforcing a series of nonlinear projections. Such projections automatically embed the discriminative cues hidden in both views into the common space, and better visual recognition is thus achieved on the test data that stems from only the main view. The efficacy of our proposed auxiliary learning approach is demonstrated through three challenging visual recognition tasks with different kinds of auxiliary information.

Convolutional Neural Network-Based Robot Navigation Using Uncalibrated Spherical Images

Vision-based mobile robot navigation is a vibrant area of research with numerous algorithms having been developed, the vast majority of which either belong to the scene-oriented simultaneous localization and mapping (SLAM) or fall into the category of robot-oriented lane-detection/trajectory tracking. These methods suffer from high computational cost and require stringent labelling and calibration efforts. To address these challenges, this paper proposes a lightweight robot navigation framework based purely on uncalibrated spherical images. To simplify the orientation estimation, path prediction and improve computational efficiency, the navigation problem is decomposed into a series of classification tasks. To mitigate the adverse effects of insufficient negative samples in the “navigation via classification” task, we introduce the spherical camera for scene capturing, which enables 360° fisheye panorama as training samples and generation of sufficient positive and negative heading directions. The classification is implemented as an end-to-end Convolutional Neural Network (CNN), trained on our proposed Spherical-Navi image dataset, whose category labels can be efficiently collected. This CNN is capable of predicting potential path directions with high confidence levels based on a single, uncalibrated spherical image. Experimental results demonstrate that the proposed framework outperforms competing ones in realistic applications.

Multi-View Visual Recognition of Imperfect Testing Data

A practical yet under-explored problem often encountered by multimedia researchers is the recognition of imperfect testing data, where multiple sensing channels are deployed but interference or transmission distortion corrupts some of them. Typical cases of imperfect testing data include missing features and feature misalignments. To address these challenges, we choose the latent space model and introduce a new similarity learning canonical-correlation analysis (SLCCA) method to capture the semantic consensus between views. The consensus information is preserved by projection matrices learned with modified canonical-correlation analysis (CCA) optimization terms with new, explicit class-similarity constraints. To make it computationally tractable, we propose to combine a practical relaxation and an alternating scheme to solve the optimization problem. Experiments on four challenging multi-view visual recognition datasets demonstrate the efficacy of the proposed method.

Auxiliary Training Information Assisted Visual Recognition

In the realm of multi-modal visual recognition, the reliability of the data acquisition system is often a concern due to the increased complexity of the sensors. One of the major issues is the accidental loss of one or more sensing channels, which poses a major challenge to current learning systems. In this paper, we examine one of these specific missing data problems, where we have a main modality/view along with an auxiliary modality/view present in the training data, but merely the main modality/view in the test data. To effectively leverage the auxiliary information to train a stronger classifier, we propose a collaborative auxiliary learning framework based on a new discriminative canonical correlation analysis. This framework reveals a common semantic space shared across both modalities/views through enforcing a series of nonlinear projections. Such projections automatically embed the discriminative cues hidden in both modalities/views into the common space, and better visual recognition is thus achieved on the test data. The efficacy of our proposed auxiliary learning approach is demonstrated through four challenging visual recognition tasks with different kinds of auxiliary information.

Fast implementation of sparse iterative covariance-based estimation for source localization

Fast implementations of the sparse iterative covariance-based estimation (SPICE) algorithm are presented for source localization with a uniform linear array (ULA). SPICE is a robust, user parameter-free, high-resolution, iterative, and globally convergent estimation algorithm for array processing. SPICE offers superior resolution and lower sidelobe levels for source localization compared to the conventional delay-and-sum beamforming method; however, a traditional SPICE implementation has a higher computational complexity (which is exacerbated in higher dimensional data). It is shown that the computational complexity of the SPICE algorithm can be mitigated by exploiting the Toeplitz structure of the array output covariance matrix using Gohberg-Semencul factorization. The SPICE algorithm is also extended to the acoustic vector-sensor ULA scenario with a specific nonuniform white noise assumption, and the fast implementation is developed based on the block Toeplitz properties of the array output covariance matrix. Finally, numerical simulations illustrate the computational gains of the proposed methods.

Fast implementation of sparse iterative covariance-based estimation for array processing

Fast implementations of the SParse Iterative Covariance-based Estimation (SPICE) algorithm are presented for source localization in passive sonar applications. SPICE is a robust, user parameter-free, high-resolution, iterative and globally convergent estimation algorithm for array processing. SPICE offers superior resolution and lower sidelobe levels for source localization at the cost of a higher computational complexity compared to the conventional delay-and-sum beamforming method. It is shown in this paper that the computational complexity of the SPICE algorithm can be reduced by exploiting the Toeplitz structure of the array output covariance matrix using the Gohberg-Semencul factorization. The fast implementations for both the hydrophone uniform linear array (ULA) and the vector-sensor ULA scenarios are proposed and the computational gains are illustrated by numerical simulations.

A Hyperspectral Image Classification Framework with Spatial Pixel Pair Features

During recent years, convolutional neural network (CNN)-based methods have been widely applied to hyperspectral image (HSI) classification by mostly mining the spectral variabilities. However, the spatial consistency in HSI is rarely discussed except as an extra convolutional channel. Very recently, the development of pixel pair features (PPF) for HSI classification offers a new way of incorporating spatial information. In this paper, we first propose an improved PPF-style feature, the spatial pixel pair feature (SPPF), that better exploits both the spatial/contextual information and spectral information. On top of the new SPPF, we further propose a flexible multi-stream CNN-based classification framework that is compatible with multiple in-stream sub-network designs. The proposed SPPF is different from the original PPF in its paring pixel selection strategy: only pixels immediately adjacent to the central one are eligible, therefore imposing stronger spatial regularization. Additionally, with off-the-shelf classification sub-network designs, the proposed multi-stream, late-fusion CNN-based framework outperforms competing ones without requiring extensive network configuration tuning. Experimental results on three publicly available datasets demonstrate the performance of the proposed SPPF-based HSI classification framework.

Attention-Based Temporal Weighted Convolutional Neural Network for Action Recognition

Research in human action recognition has accelerated significantly since the introduction of powerful machine learning tools such as Convolutional Neural Networks (CNNs). However, effective and efficient methods for incorporation of temporal information into CNNs are still being actively explored in the recent literature. Motivated by the popular recurrent attention models in the research area of natural language processing, we propose the Attention-based Temporal Weighted CNN (ATW), which embeds a visual attention model into a temporal weighted multi-stream CNN. This attention model is simply implemented as temporal weighting yet it effectively boosts the recognition performance of video representations. Besides, each stream in the proposed ATW framework is capable of end-to-end training, with both network parameters and temporal weights optimized by stochastic gradient descent (SGD) with backpropagation. Our experiments show that the proposed attention mechanism contributes substantially to the performance gains with the more discriminative snippets by focusing on more relevant video segments.

Segment-Tube: Spatio-Temporal Action Localization in Untrimmed Videos with Per-Frame Segmentation

Inspired by the recent spatio-temporal action localization efforts with tubelets (sequences of bounding boxes), we present a new spatio-temporal action localization detector Segment-tube, which consists of sequences of per-frame segmentation masks. The proposed Segment-tube detector can temporally pinpoint the starting/ending frame of each action category in the presence of preceding/subsequent interference actions in untrimmed videos. Simultaneously, the Segment-tube detector produces per-frame segmentation masks instead of bounding boxes, offering superior spatial accuracy to tubelets. This is achieved by alternating iterative optimization between temporal action localization and spatial action segmentation. Experimental results on three datasets validated the efficacy of the proposed method, including (1) temporal action localization on the THUMOS 2014 dataset; (2) spatial action segmentation on the Segtrack dataset; and (3) joint spatio-temporal action localization on the newly proposed ActSeg dataset. It is shown that our method compares favorably with existing state-of-the-art methods.