Songhe Feng

A Self-immunizing Manifold Ranking for Image Retrieval

Manifold ranking (MR), as a powerful semi-supervised learning algorithm, plays an important role to deal with the relevance feedback problem in content-based image retrieval (CBIR). However, conventional MR has two main drawbacks: 1) in many cases, it is prone to exploit “unreliable” unlabeled images when deployed in CBIR due to the semantic gap; 2) the performance of MR is quite sensitive to the scale parameter used for calculating the Laplacian matrix. In this work, a self-immunizing MR approach is presented to address the drawbacks. Concretely, we first propose an elastic kNN graph as well as its constructing algorithm to exploit unlabeled images “safely”, and then develop a local scaling solution to calculate the Laplacian matrix adaptively. Extensive experiments on 10,000 Corel images show that the proposed algorithm is more effective than the state-of-the-art approaches.

From sample selection to model update

This paper proposes an online tracking algorithm that employs a confidence combinatorial map model. Drifting is a problem that easily occurs in object tracking and most of the recent tracking algorithms have attempted to solve this problem. In this paper, we propose a confidence combinatorial map that describes the structure of the object, based on which the confidence combinatorial map model is developed. The model associates the relationship between the object in the current frame and that in the previous frame. On the strength of this relationship, more precisely classified samples can be selected and are employed in the model update stage, which directly influences the occurrence of the tracking drift. The proposed algorithm was estimated on several public video sequences and the performance was compared with several state-of-the-art algorithms. The experiments demonstrate that the proposed algorithm outperforms other comparative algorithms and gives a very good performance.

An error-tolerant approximate matching algorithm for labeled combinatorial maps

Combinatorial maps are widely used in image representation and processing, and measuring distance or similarity between combinatorial maps is therefore an important issue in this field. The existed distance measures between combinatorial maps based on the largest common submap and the edit distance have high computational complexity, and are hard to be applied in real applications. This paper addresses the problem of inexact matching between labeled combinatorial maps, and aims to find a rapid algorithm for measuring distance between maps. We first define joint-tree of combinatorial maps and prove that it can be used to decide of isomorphism between combinatorial maps. Subsequently, a distance measure based on joint-trees and an approximate approach are proposed to compute the distance between combinatorial maps. Experimental results show that the proposed approach performs better in practice than the previous approach based on approximate map edit distance.

A framework with a multi-task CNN model joint with a re-ranking method for vehicle re-identification

Recently, with the number of surveillance cameras growing rapidly, vehicle re-identification (re-id) plays a major part in the traffic surveillance. Most exiting methods for vehicle re-id are mainly focused on single convolutional neural network (CNN) model i.e. identification or verification model to extract features. However, single model has their own drawbacks and it can not extract enough discriminative feature. In this work, we propose a new framework with a multi-task CNN model and a ranking optimization method to tackle the re-id task. The multi-task CNN model combines the strengths of the two models, which can get a more discriminative feature of vehicle image. The ranking optimization method utilizes the relationship of the k-nearest neighbors of the probe and the gallery image to optimize the final ranking list. Experiments are carried out to demonstrate that the good performance of our framework with improvements of 1.5% and 20% at the top-1 ranking matching accuracy on two mainstream datasets VeRi and VehicleID.

Recurrent convolutional network for video-based smoke detection

Video-based smoke detection plays an important role in the fire detection community. Such interesting topic, however, always suffers from great challenge due to the large variances of smoke texture, shape and color in the real applications. To effectively exploiting the long-range motion context, we propose a novel video-based smoke detection method via Recurrent Neural Networks (RNNs). More concretely, the proposed method first captures the space and motion context information by using deep convolutional motion-space networks. Then a temporal pooling layer and RNNs are used to effectively train the smoke model. Finally, to promote further research and evaluation of video-based smoke models, we also construct a new large database of 3000 challenging smoke video clips that cover large variations in illuminance and weather conditions. Experimental results demonstrate that our proposed method is capable of achieving state-of-the-art performance on all public benchmarks.

Semi-supervised dual low-rank feature mapping for multi-label image annotation

Automatic image annotation as a typical multi-label learning problem, has gained extensive attention in recent years owing to its application in image semantic understanding and relevant disciplines. Nevertheless, existing annotation methods share the same challenge that labels annotated on the training images are usually incomplete and unclean, while the need for adequate training data is costly and unrealistic. Being aware of this, we propose a dual low-rank regularized multi-label learning model under a graph regularized semi-supervised learning framework, which can effectively capture the label correlations in the learned feature space, and enforce the label matrix be self-recovered in label space as well. To be specific, the proposed approach firstly puts forward a label matrix refinement approach, by introducing a label coefficient matrix to build a linear self-recovery model. Then, graph Laplacian regularization is introduced to make use of a large number of unlabeled images by enforcing the local geometric structure on both labeled and unlabeled images. Lastly, we exploit dual trace norm regularization on both feature mapping matrix and self-recovery coefficient matrix to capture the correlations among different labels in both feature space and label space, and control the model complexity as well. Empirical studies on four real-world image datasets demonstrate the effectiveness and efficiency of the proposed framework.

Saliency Ranker: A New Salient Object Detection Method

Abstract Recently, saliency detection has become an active research topic in learning from labeled image, where various supervised methods were designed. Many existing methods usually cast saliency detection as a binary classification or regression problem, in which saliency detection performance relies heavily on the expensive pixel-wise annotations of salient objects. This paper addresses the issue by developing a novel learning-to-rank model with a limited number of training data, which combines the strength of cost-sensitive label ranking methods with the power of low-rank matrix recovery theories. Rather than using a binary decision for each saliency value, our approach ranks saliency values in a descending order with the estimated relevance to the given saliency. Additionally, we also aggregate the prediction models for different saliency labels into a matrix, and solve saliency ranking via a low-rank matrix recovery problem. Extensive experiments over challenging benchmarks clearly validate advantage of our method.

A novel hypergraph matching algorithm based on tensor refining

Abstract Hypergraph matching utilizes high order constraints rather than unary or pairwise ones, which aims to establish a more reliable correspondence between two sets of image features. Although many hypergraph matching methods have been put forward over the past decade, it remains a challenging problem to be solved due to its combinatorial nature. Most of these methods are based on tensor marginalization, where tensor entries representing joint probabilities of the assignment are fixed during the iterations meanwhile the individual assignment probabilities evolving. This will cause some incomplete information which may hurt the matching performance. Addressing this issue, we propose a novel hypergraph matching algorithm based on tensor refining, accompanied with an alternative adjustment method to accelerate the convergence. We make a comparison between the proposed approach and several outstanding matching algorithms on three commonly used benchmarks. The experimental results validate the superiority of our method on both matching accuracy and robustness against noise and deformation.

Joint Tree of Combinatorial Maps

Combinatorial maps are widely used in the field of computer vision, including image segmentation, medical image analysis and mobile robotics. Many practical problems can be formulated as the combinatorial map matching problem. This paper addresses the problem of inexact matching between labeled combinatorial maps. We define Joint Tree of combinatorial maps, and prove it can be used to decide of map isomorphism. In this way, the map matching problem is relaxed to the Joint Tree matching problem, which can be solved in polynomial time. Our approach provides a novel way to explore the problem of combinatorial map matching.