Zhi Zeng

Adaptive bit allocation hashing for approximate nearest neighbor search

Abstract Using hashing algorithms to learn binary codes representation of data for fast approximate nearest neighbor (ANN) search has attracted more and more attention. Most existing hashing methods employ various hash functions to encode data. The resulting binary codes can be obtained by concatenating bits produced by those hash functions. These methods usually have two main steps: projection and thresholding. One problem with these methods is that every dimension of the projected data is regarded as of same importance and encoded by one bit, which may result in ineffective codes. In this paper, we introduce an adaptive bit allocation hashing (ABAH) method to encode data for ANN search. The basic idea is, according to the dispersions of all the dimensions after projection we use different numbers of bits to encode them. In our method, more bits will be adaptively allocated to encode dimensions with larger dispersion while fewer bits for dimensions with smaller dispersion. This novel bit allocation scheme makes our hashing method effectively preserve the neighborhood structure in the original data space. Extensive experiments show that the proposed ABAH significantly outperforms other state-of-the-art methods for ANN search task.

A novel geometrically invariant blind robust watermarking algorithm based on SVD and DCT

In this paper, we propose a novel blind image watermarking algorithm that is resistant to various attacks, including geometrical distortions. To embed the watermark, we first enlarge the image with a small scale to be fit for the next image blocking process. Secondly, the enlarged image is blocked into macroblocks and subblocks with adaptively determined sizes. Third, we scramble the watermark and embed it repeatedly into the coefficients that are obtained from the subblock-based SVD followed by macroblock-based DCT. Then, the watermarked image can be obtained by means of corresponding inverse transformations. The watermark extraction process is similar as the embedding one. Compared with others works, the small-scale image enlargement, the adaptive image blocking and the watermark scrambling we present in this SVD-DCT based method play an important role in increasing the robustness to resist geometrical distortions. Experimental results demonstrate that our method can not only resist common image processing attacks, but also has good performance when the image has been scaled or cropped.

Adaptive bit allocation product quantization

Product quantization (PQ) is a popular vector quantization method for approximate nearest neighbor search. The key idea of PQ is to decompose the original data space into the Cartesian product of some low-dimensional subspaces and then every subspace is quantized separately with the same number of codewords. However, the performance of PQ depends largely on the distribution of the original data. If the energies of subspaces are extremely unbalanced, PQ will achieve bad results. In this paper, we propose an adaptive bit allocation product quantization (BAPQ) method to deal with the problem of unbalanced energies of subspaces in PQ. In BAPQ, we adaptively allocate different numbers of codewords (or bits) to subspaces to quantize data for minimizing the total quantization distortion. The essence of our method is to find the optimal bit allocation scheme. To this end, we formulate an objective function about minimizing quantization distortion with respect to bit allocation scheme and adopt a greedy algorithm to find the near-optimal solution. BAPQ can achieve lower quantization distortion than PQ and optimized product quantization (OPQ). Besides, both bias and variance of difference between the true distance and the BAPQs estimated distance are reduced from those of PQ and OPQ. Extensive experiments have verified the superiority of BAPQ over state-of-the-art approaches.

A hierarchical generative model for Generic Audio Document Categorization

In this paper, we call the pattern classification problem that consists in assigning a category label to a long audio signal based on its semantic content as Generic Audio Document Categorization (GADC). A novel generative model is proposed to describe the generic audio document categories and solve the GADC problem. This model is a four-level hierarchical model in which two latent variables “audio topic” and “audio word” are introduced in addition to the two observed variables category and audio feature. We present an iterative learning algorithm including two Expectation-Maximization (EM) cycles to estimate the model parameters and give a discriminative document weighting procedure to make the model more discriminative. Subsequently, the distribution of “audio topic” in the well-trained model is utilized to represent each generic audio document category. This is same with some bag-of-word methods. However, our method is advanced since it does not require quantizing the continuous audio features to a vocabulary of “audio words”. Finally, experiment results show the effectiveness of our approach.

SIFT Matching with CNN Evidences for Particular Object Retrieval

Abstract Many object instance retrieval systems are typically based on matching of local features, such as SIFT. However, these local descriptors serve as low-level clues, which are not sufficiently distinctive to prevent false matches. Recently, deep convolutional neural networks (CNN) have shown their promise as a semantic-aware representation for many computer vision tasks. In this paper, we propose a novel approach to employ CNN evidences to improve the SIFT matching accuracy, which plays a critical role in improving the object retrieval performance. To weaken the interference of noise, we extract compact CNN representations from a number of generic object regions. Then a query-adaptive method is proposed to choose appropriate CNN evidence to verify each pre-matched SIFT pair. Two different visual matching verification functions are introduced and evaluated. Moreover, we investigate the suitability of fine-tuning the CNN for our proposed approach. Extensive experiments on benchmark datasets demonstrate the effectiveness of our method for particular object retrieval. Our results compare favorably to the state-of-the-art methods with acceptable memory usage and query time.

Transmitting informative components of fisher codes for mobile visual search

Existing techniques usually adopt compact descriptors such as Fisher vector for mobile visual search, since compact descriptors are memory-efficient and suitable for fast transmission. In common Fisher vector methods, in order to make the size of image representations small enough for efficient transmission, only a small number of visual words are used. However, this choice usually sacrifices the search accuracy. In this paper, a Soft-Assignment Adjusting approach is proposed to just select informative components of descriptors for query. With this method, we can adopt more visual words to improve accuracy, while the memory usage is still low. Furthermore, efficient bitrate scalable codes are proposed in order to accommodate the network bandwidth variation. Experiments performed on benchmark datasets show that our proposed approach outperforms the state-of-the-art methods for mobile visual search.

Region matching and similarity enhancing for image retrieval

Many image retrieval systems adopt the bag-of-words model and rely on matching of local descriptors. However, these descriptors of keypoints, such as SIFT, may lead to false matches, since they do not consider the contextual information of the keypoints. In this paper, we incorporate the cues of meaningful regions where local descriptors are extracted. We describe a matching region estimation (MRE) method to find appropriate matching regions for local descriptor matching pairs. Then the region matching quality is evaluated and the true matched regions will enhance the similarity of local descriptors. Consequently, the image retrieval accuracy can be improved. Extensive experiments on benchmark datasets show the effectiveness of our method and our result compares favorably with the state-of-the-art.

Product image search with regional evidences

In the task of product image search, the database consists of clean versions of product images, while the query photos are often captured from mobile phone cameras under uncontrolled conditions. Conventional methods usually adopt the SIFT based bag-of-words (BoW) representation of the whole query image, which suffers from the interference of background noise. To address the problem, we extract multiple candidate regions from the query image and compute the regional similarity to database images individually. Then a verification strategy is proposed to evaluate the similarity based on regional semantic evidences. With the proposed method, we can not only improve the search accuracy, but also obtain the location of the product in the query image. Extensive experiments on two public datasets demonstrate the effectiveness of our method.

Uniform Variance Product Quantization

Product quantization (PQ) is an efficient and effective vector quantization approach to fast approximate nearest neighbor (ANN) search especially for high-dimensional data. The basic idea of PQ is to decompose the original data space into the Cartesian product of some low-dimensional subspaces and then every subspace is quantized separately with the same number of codewords. However, the performance of PQ depends largely on the distribution of the original data. If the distributions of every subspace have larger difference, PQ will achieve bad results as shown in our experiments. In this paper, we propose a uniform variance product quantization (UVPQ) scheme to project the data by a uniform variance projection before decompose it, which can minimize the subspace distribution difference of the whole space. UVPQ can guarantee good results however the data rotate. Extensive experiments have verified the superiority of UVPQ over PQ for ANN search.

Optimized K-Means Hashing for Approximate Nearest Neighbor Search

Hashing which maps data into binary codes in Hamming space has attracted more and more attentions for approximate nearest neighbor search due to its high efficiency and reduced storage cost. K-means hashing (KH) is a novel hashing method which firstly quantizes the data by codewords and then uses the indices of codewords to encode the data. However, in KH, only the codewords are updated to minimize the quantization error and affinity error while the indices of codewords remain the same after they are initialized. In this paper, we propose an optimized k-means hashing (OKH) method to encode data by binary codes. In our method, we simultaneously optimize the codewords and the indices of them to minimize the quantization error and the affinity error. Our OKH method can find both the optimal codewords and the optiaml indices, and the resulting binary codes in Hamming space can better preserve the original neighborhood structure of the data. Besides, OKH can further be generalized to a product space. Extensive experiments have verified the superiority of OKH over KH and other state-of-the-art hashing methods.