Mingyi He

A Simple Prior-Free Method for Non-rigid Structure-from-Motion Factorization

This paper proposes a simple “prior-free” method for solving the non-rigid structure-from-motion (NRSfM) factorization problem. Other than using the fundamental low-order linear combination model assumption, our method does not assume any extra prior knowledge either about the non-rigid structure or about the camera motions. Yet, it works effectively and reliably, producing optimal results, and not suffering from the inherent basis ambiguity issue which plagued most conventional NRSfM factorization methods. Our method is very simple to implement, which involves solving a very small SDP (semi-definite programming) of fixed size, and a nuclear-norm minimization problem. We also present theoretical analysis on the uniqueness and the relaxation gap of our solutions. Extensive experiments on both synthetic and real motion capture data (assuming following the low-order linear combination model) are conducted, which demonstrate that our method indeed outperforms most of the existing non-rigid factorization methods. This work offers not only new theoretical insight, but also a practical, everyday solution to NRSfM.

Band selection based on feature weighting for classification of hyperspectral data

A new feature weighting method for band selection is presented, which is based on the pairwise separability criterion and matrix coefficients analysis. Through decorrelation of each class by principal component transformation, the criterion value of any band subset is the summations of the values of individual bands of it for the transformed feature space, and thus the computation amounts of calculating criteria of each band combinations are reduced. Following it, the corresponding matrix coefficients analysis is done to assign weights to original bands. As feature weighting considers little about the spectral correlation, the redundant bands are removed by choosing those with lower correlation coefficients than a preset threshold. Hyperspectral data classification experiments show the effectiveness of the new band selection method.

Depth and surface normal estimation from monocular images using regression on deep features and hierarchical CRFs

Predicting the depth (or surface normal) of a scene from single monocular color images is a challenging task. This paper tackles this challenging and essentially underdetermined problem by regression on deep convolutional neural network (DCNN) features, combined with a post-processing refining step using conditional random fields (CRF). Our framework works at two levels, super-pixel level and pixel level. First, we design a DCNN model to learn the mapping from multi-scale image patches to depth or surface normal values at the super-pixel level. Second, the estimated super-pixel depth or surface normal is refined to the pixel level by exploiting various potentials on the depth or surface normal map, which includes a data term, a smoothness term among super-pixels and an auto-regression term characterizing the local structure of the estimation map. The inference problem can be efficiently solved because it admits a closed-form solution. Experiments on the Make3D and NYU Depth V2 datasets show competitive results compared with recent state-of-the-art methods.

A simple prior-free method for non-rigid structure-from-motion factorization

This paper proposes a simple “prior-free” method for solving non-rigid structure-from-motion factorization problems. Other than using the basic low-rank condition, our method does not assume any extra prior knowledge about the nonrigid scene or about the camera motions. Yet, it runs reliably, produces optimal result, and does not suffer from the inherent basis-ambiguity issue which plagued many conventional nonrigid factorization techniques. Our method is easy to implement, which involves solving no more than an SDP (semi-definite programming) of small and fixed size, a linear Least-Squares or trace-norm minimization. Extensive experiments have demonstrated that it outperforms most of the existing linear methods of nonrigid factorization. This paper offers not only new theoretical insight, but also a practical, everyday solution, to non-rigid structure-from-motion.

Hyperspectral Image Resolution Enhancement Using High-Resolution Multispectral Image Based on Spectral Unmixing

In this paper, a hyperspectral (HS) image resolution enhancement algorithm based on spectral unmixing is proposed for the fusion of the high-spatial-resolution multispectral (MS) image and the low-spatial-resolution HS image (HSI). As a result, a high-spatial-resolution HSI is reconstructed based on the high spectral features of the HSI represented by endmembers and the high spatial features of the MS image represented by abundances. Since the number of endmembers extracted from the MS image cannot exceed the number of bands in least-squares-based spectral unmixing algorithm, large reconstruction errors will occur for the HSI, which degrades the fusion performance of the enhanced HSI. Therefore, in this paper, a novel fusion framework is also proposed by dividing the whole image into several subimages, based on which the performance of the proposed spectral-unmixing-based fusion algorithm can be further improved. Finally, experiments on the Hyperspectral Digital Imagery Collection Experiment and Airborne Visible/Infrared Imaging Spectrometer data demonstrate that the proposed fusion algorithms outperform other famous fusion techniques in both spatial and spectral domains.

Classification Based on 3-D DWT and Decision Fusion for Hyperspectral Image Analysis

In this letter, a fusion-classification system is proposed to alleviate ill-conditioned distributions in hyperspectral image classification. A windowed 3-D discrete wavelet transform is first combined with a feature grouping-a wavelet-coefficient correlation matrix (WCM)-to extract and select spectral-spatial features from the hyperspectral image dataset. The adjacent wavelet-coefficient subspaces (from the WCM) are intelligently grouped such that correlated coefficients are assigned to the same group. Afterwards, a multiclassifier decision-fusion approach is employed for the final classification. The performance of the proposed classification system is assessed with various classifiers, including maximum-likelihood estimation, Gaussian mixture models, and support vector machines. Experimental results show that with the proposed fusion system, independent of the classifier adopted, the proposed classification system substantially outperforms the popular single-classifier classification paradigm under small-sample-size conditions and noisy environments.

Element-wise factorization for N-View projective reconstruction

Sturm-Triggs iteration is a standard method for solving the projective factorization problem. Like other iterative algorithms, this method suffers from some common drawbacks such as requiring a good initialization, the iteration may not converge or only converge to a local minimum, etc. None of the published works can offer any sort of global optimality guarantee to the problem. In this paper, an optimal solution to projective factorization for structure and motion is presented, based on the same principle of low-rank factorization. Instead of formulating the problem as matrix factorization, we recast it as element-wise factorization, leading to a convenient and efficient semi-definite program formulation. Our method is thus global, where no initial point is needed, and a globally-optimal solution can be found (up to some relaxation gap). Unlike traditional projective factorization, our method can handle real-world difficult cases like missing data or outliers easily, and all in a unified manner. Extensive experiments on both synthetic and real image data show comparable or superior results compared with existing methods.

Projective Multiview Structure and Motion from Element-Wise Factorization

The Sturm-Triggs type iteration is a classic approach for solving the projective structure-from-motion (SfM) factorization problem, which iteratively solves the projective depths, scene structure, and camera motions in an alternated fashion. Like many other iterative algorithms, the Sturm-Triggs iteration suffers from common drawbacks, such as requiring a good initialization, the iteration may not converge or may only converge to a local minimum, and so on. In this paper, we formulate the projective SfM problem as a novel and original element-wise factorization (i.e., Hadamard factorization) problem, as opposed to the conventional matrix factorization. Thanks to this formulation, we are able to solve the projective depths, structure, and camera motions simultaneously by convex optimization. To address the scalability issue, we adopt a continuation-based algorithm. Our method is a global method, in the sense that it is guaranteed to obtain a globally optimal solution up to relaxation gap. Another advantage is that our method can handle challenging real-world situations such as missing data and outliers quite easily, and all in a natural and unified manner. Extensive experiments on both synthetic and real images show comparable results compared with the state-of-the-art methods.

Optimizing Hopfield Neural Network for Spectral Mixture Unmixing on GPU Platform

The Hopfield neural network (HNN) has been demonstrated to be an effective tool for the spectral mixture unmixing of hyperspectral images. However, it is extremely time consuming for such per-pixel algorithm to be utilized in real-world applications. In this letter, the implementation of a multichannel structure of HNN (named as MHNN) on a graphics processing unit (GPU) platform is proposed. According to the unmixing procedure of MHNN, three levels of parallelism, including thread, block, and stream, are designed to explore the peak computing capacity of a GPU device. In addition, constant and texture memories are utilized to further improve its computational performance. Experiments on both synthetic and real hyperspectral images demonstrated that the proposed GPU-based implementation works on the peak computing ability of a GPU device and obtains several hundred times of acceleration versus the CPU-based implementation while its unmixing performance remains unchanged.

A New Adaptive Algorithm for Despeckling SAR Images Based on Contourlet Transform

A new adaptive contourlet transform-based technique for SAR image speckles removal is presented. It relies on realistic distribution of the SAR images contourlet coefficients that represent mainly speckle noise on one hand and those that represent the useful signals corrupted on the other hand. The analytic model for these distributions is proposed. The shrinkage factor for despeckling is estimated at neighbourhood of the reference contourlet coefficient in each sub bands. Finally, a comparison of performance of Lee filter, Forster filter, Gamma filter, wavelet-based despeckling and contourlet transform-based despeckling is provided for both simulated and actual SAR images. It shows that the contourlet methods strongly suppress speckle, while preserving image details and sharpness