Zhi-Yong Liu

GNCCP—Graduated NonConvexityand Concavity Procedure

In this paper we propose the graduated nonconvexity and concavity procedure (GNCCP) as a general optimization framework to approximately solve the combinatorial optimization problems defined on the set of partial permutation matrices. GNCCP comprises two sub-procedures, graduated nonconvexity which realizes a convex relaxation and graduated concavity which realizes a concave relaxation. It is proved that GNCCP realizes exactly a type of convex-concave relaxation procedure (CCRP), but with a much simpler formulation without needing convex or concave relaxation in an explicit way. Actually, GNCCP involves only the gradient of the objective function and is therefore very easy to use in practical applications. Two typical related NP-hard problems, partial graph matching and quadratic assignment problem (QAP), are employed to demonstrate its simplicity and state-of-the-art performance.In this paper we propose the graduated nonconvexity and concavity procedure (GNCCP) as a general optimization framework to approximately solve the combinatorial optimization problems defined on the set of partial permutation matrices. GNCCP comprises two sub-procedures, graduated nonconvexity which realizes a convex relaxation and graduated concavity which realizes a concave relaxation. It is proved that GNCCP realizes exactly a type of convex-concave relaxation procedure (CCRP), but with a much simpler formulation without needing convex or concave relaxation in an explicit way. Actually, GNCCP involves only the gradient of the objective function and is therefore very easy to use in practical applications. Two typical related NP-hard problems, partial graph matching and quadratic assignment problem (QAP), are employed to demonstrate its simplicity and state-of-the-art performance.

One-Bit-Matching Conjecture for Independent Component Analysis

The one-bit-matching conjecture for independent component analysis (ICA) could be understood from different perspectives but is basically stated as all the sources can be separated as long as there is a one-toone same-sign-correspondence between the kurtosis signs of all source probability density functions (pdfs) and the kurtosis signs of all model pdfs (Xu, Cheung, & Amari, 1998a). This conjecture has been widely believed in the ICA community and implicitly supported by many ICA studies, such as the Extended Infomax (Lee, Girolami, & Sejnowski, 1999) and the soft switching algorithm (Welling & Weber, 2001). However, there is no mathematical proof to confirm the conjecture theoretically. In this article, only skewness and kurtosis are considered, and such a mathematical proof is given under the assumption that the skewness of the model densities vanishes. Moreover, empirical experiments are demonstrated on the robustness of the conjecture as the vanishing skewness assumption breaks. As a by-product, we also show that the kurtosis maximization criterion (Moreau & Macchi, 1996) is actually a special case of the minimum mutual information criterion for ICA.

An Extended Path Following Algorithm for Graph-Matching Problem

The path following algorithm was proposed recently to approximately solve the matching problems on undirected graph models and exhibited a state-of-the-art performance on matching accuracy. In this paper, we extend the path following algorithm to the matching problems on directed graph models by proposing a concave relaxation for the problem. Based on the concave and convex relaxations, a series of objective functions are constructed, and the Frank-Wolfe algorithm is then utilized to minimize them. Several experiments on synthetic and real data witness the validity of the extended path following algorithm.

Multiple ellipses detection in noisy environments: A hierarchical approach

Detection of multiple ellipses in noisy environments is a basic yet challenging task in many vision related problems. The key area of difficulty is on distinguishing the pixels pertaining to each target in the presence of noise. To tackle with the issue, we propose a hierarchical approach which is motivated by the fact that any segment of an ellipse can identify itself in ellipse reconstruction. First, we find all the neat edges without any branches, followed by an ellipse fitting on each of them. Second, some target candidates are estimated based on the neat edges, by a proposed grouping strategy. Finally, the targets are detected based on the candidates, by a proposed selective competitive algorithm to distinguish the true pixels of each target. A real application of the proposed method is illustrated in addition to some other demonstrative experiments.

Improved system for object detection and star/galaxy classification via local subspace analysis

The two traditional tasks of object detection and star/galaxy classification in astronomy can be automated by neural networks because the nature of the problems is that of pattern recognition. A typical existing system can be further improved by using one of the local Principal Component Analysis (PCA) models. Our analysis in the context of object detection and star/galaxy classification reveals that local PCA is not only superior to global PCA in feature extraction, but is also superior to gaussian mixture in clustering analysis. Unlike global PCA which performs PCA for the whole data set, local PCA applies PCA individually to each cluster of data. As a result, local PCA often outperforms global PCA for data of multi-modes. Moreover, since local PCA can effectively avoid the trouble of having to specify a large number of free elements of each covariance matrix of gaussian mixture, it can give a better description of local subspace structures of each cluster when applied on high dimensional data with small sample size. In this paper, the local PCA model proposed by Xu [IEEE Trans. Neural Networks 12 (2001) 822] under the general framework of Bayesian Ying Yang (BYY) normalization learning will be adopted. Endowed with the automatic model selection ability of BYY learning, the BYY normalization learning-based local PCA model can cope with those object detection and star/galaxy classification tasks with unknown model complexity. A detailed algorithm for implementation of the local PCA model will be proposed, and experimental results using both synthetic and real astronomical data will be demonstrated.

Strip line detection and thinning by RPCL-based local PCA

We solve the tasks of strip line detection and thinning in image processing and pattern recognition with the help of a statistical learning technique called rival penalized competitive learning based local principal component analysis. Due to its model selection and noise resistance ability, the technique is experimentally shown to outperform conventional Hough transform and thinning algorithms.

Online Multi-Modal Distance Metric Learning with Application to Image Retrieval

Distance metric learning (DML) is an important technique to improve similarity search in content-based image retrieval. Despite being studied extensively, most existing DML approaches typically adopt a single-modal learning framework that learns the distance metric on either a single feature type or a combined feature space where multiple types of features are simply concatenated. Such single-modal DML methods suffer from some critical limitations: (i) some type of features may significantly dominate the others in the DML task due to diverse feature representations; and (ii) learning a distance metric on the combined high-dimensional feature space can be extremely time-consuming using the naive feature concatenation approach. To address these limitations, in this paper, we investigate a novel scheme of online multi-modal distance metric learning (OMDML), which explores a unified two-level online learning scheme: (i) it learns to optimize a distance metric on each individual feature space; and (ii) then it learns to find the optimal combination of diverse types of features. To further reduce the expensive cost of DML on high-dimensional feature space, we propose a low-rank OMDML algorithm which not only significantly reduces the computational cost but also retains highly competing or even better learning accuracy. We conduct extensive experiments to evaluate the performance of the proposed algorithms for multi-modal image retrieval, in which encouraging results validate the effectiveness of the proposed technique.

Rapid and brief communication: Multisets mixture learning-based ellipse detection

We develop an ellipse detection algorithm based on the multisets mixture learning (MML) that differs from the conventional Hough transform perspective. The algorithm developed has potential advantages in terms of noise resistance, incomplete ellipse detection, and detecting a multitude of ellipses.

Graph Matching by Simplified Convex-Concave Relaxation Procedure

The convex and concave relaxation procedure (CCRP) was recently proposed and exhibited state-of-the-art performance on the graph matching problem. However, CCRP involves explicitly both convex and concave relaxations which typically are difficult to find, and thus greatly limit its practical applications. In this paper we propose a simplified CCRP scheme, which can be proved to realize exactly CCRP, but with a much simpler formulation without needing the concave relaxation in an explicit way, thus significantly simplifying the process of developing CCRP algorithms. The simplified CCRP can be generally applied to any optimizations over the partial permutation matrix, as long as the convex relaxation can be found. Based on two convex relaxations, we obtain two graph matching algorithms defined on adjacency matrix and affinity matrix, respectively. Extensive experimental results witness the simplicity as well as state-of-the-art performance of the two simplified CCRP graph matching algorithms.

Online multi-task collaborative filtering for on-the-fly recommender systems

Traditional batch model-based Collaborative Filtering (CF) approaches typically assume a collection of users rating data is given a priori for training the model. They suffer from a common yet critical drawback, i.e., the model has to be re-trained completely from scratch whenever new training data arrives, which is clearly non-scalable for large real recommender systems where users rating data often arrives sequentially and frequently. In this paper, we investigate a novel efficient and scalable online collaborative filtering technique for on-the-fly recommender systems, which is able to effectively online update the recommendation model from a sequence of rating observations. Specifically, we propose a family of online multi-task collaborative filtering (OMTCF) algorithms, which tackle the online collaborative filtering task by exploiting the similar principle as online multitask learning. Encouraging empirical results on large-scale datasets showed that the proposed technique is significantly more effective than the state-of-the-art algorithms.