Nguyen Anh Tu

Topic modeling and improvement of image representation for large-scale image retrieval

In this paper, we present a new visual search system for finding similar images in a large database. However, there are a number of challenges regarding the robustness of the image representations and the efficiency of the retrieval framework. To tackle these challenges, we first propose an encoding technique based on soft-assignment of local features to convert an entire image into a single vector, which is a compact and discriminative representation. This encoded vector is suitable for most types of efficient indexing methods to produce an initial result. To compensate for the lack of incorporating geometric and object-related information during the encoding scheme, we then propose a probabilistic topic model to formalize the spatial structure among the local features. Moreover, the topic model allows us to effectively extract the object and background regions from the image. This is performed by a Markov Chain Monte Carlo algorithm for approximate inference. Finally, benefiting from the extracted objects in each image, we present a re-ranking scheme to automatically refine the initial search results. Our proposed retrieval framework has two major advantages: i) an aggregation strategy through soft-assignment improves the discriminative power of the representation, which has a determinative effect on the retrieval precision; and ii) the probabilistic latent topic model enables us to not only gain insight into the spatial structure of the image, but also handle a large variation in the object appearance. The experimental results from four benchmark datasets show that our approach provides competitive accuracy, and runs about ten times faster. Our studies also verify that proposed approach works effectively on large-scale databases of millions of images.

Selective bit embedding scheme for robust blind color image watermarking

In this paper, we propose a novel robust blind color image watermarking method, namely SMLE, that allows to embed a gray-scale image as watermark into a host color image in the wavelet domain. After decomposing the gray-scale watermark to component binary images in digits ordering from least significant bit (LSB) to most significant bit (MSB), the retrieved binary bits are then embedded into wavelet blocks of two optimal color channels by using an efficient quantization technique, where the wavelet coefficient difference in each block is quantized to either two pre-defined thresholds for corresponding 0-bits and 1-bits. To optimize the watermark imperceptibility, we equally split the coefficient modified quantity on two middle-frequency sub-bands instead of only one as in existing approaches. The improvement of embedding rule increases approximately 3 dB of watermarked image quality. An adequate trade-off between robustness and imperceptibility is controlled by a factor representing the embedding strength. As for extraction process, we exploit 2D Otsu algorithm for higher accuracy of watermark detection than that of 1D Otsu. Experimental results prove the robustness of our SMLE watermarking model against common image processing operations along with its efficient retention of the imperceptibility of the watermark in the host image. Compared to state-of-the-art methods, our approach outperforms in most of robustness tests at a same high payload capacity.

iTri: Index-based triangle listing in massive graphs

Abstract Triangle listing is a basic operator when dealing with many graph problems. However, in-memory algorithms do not work well with recently developed massive graphs such as social networks because these graphs cannot be accommodated in the memory. Thus, external memory-based algorithms have been proposed recently, but these approaches still require frequent multiple scans of the whole graph on the disk and large volumes of calculations are performed that involve the whole graph during every iteration. In this study, we propose a novel index-based method for listing triangles in massive graphs. First, we present new notions for the vertex range index and potential cone vertex index. Next, we propose an index join-based triangle listing algorithm. Our method accesses the indexed data asynchronously and joins them to list triangles using a multi-threaded parallel processing technique. Based on experiments, we demonstrate that our algorithm outperforms the state-of-the-art solution methods by three to eight times in terms of the wall clock time.

Hierarchical Topic Modeling With Pose-Transition Feature For Action Recognition Using 3D Skeleton Data

Abstract Despite impressive achievements in image processing and artificial intelligence in the past decade, understanding video-based action remains a challenge. However, the intensive development of 3D computer vision in recent years has brought more potential research opportunities in pose-based action detection and recognition. Thanks to the advantages of depth camera devices like the Microsoft Kinect sensor, we developed an effective approach to in-depth analysis of indoor actions using skeleton information, in which skeleton-based feature extraction and topic model-based learning are two major contributions. Geometric features, i.e. joint distance, joint angle, and joint-plane distance are calculated in the spatio-temporal dimension. These features are merged into two types, called pose and transition features, and then are provided to codebook construction to convert sparse features into visual words by k-means clustering. An efficient hierarchical model is developed to describe the full correlation of feature - poselet - action based on Pachinko Allocation Model. This model has the potential to uncover more hidden poselets, which have been recognized as the valuable information and help to differentiate pose-sharing actions. The experimental results on several well-known datasets, such as MSR Action 3D, MSR Daily Activity 3D, Florence 3D Action, UTKinect-Action 3D, and NTU RGB+D Action Recognition, demonstrate the high recognition accuracy of the proposed method. Our method outperforms state-of-the-art methods in the field in most dataset benchmarks.

Semantic image retrieval using correspondence topic model with background distribution

Social image search becomes an active research field in recent years due to the rapid development in big data processing technologies. In the retrieval systems, text description/tags play a key role to bridge the semantic gap between low-level features and higher-level concepts, and so guarantee the reliable search. However, in practice manual tags are usually noisy and incomplete, resulting in a limited performance of image retrieval. To tackle this problem, we propose a probabilistic topic model to formalize the correlation of tags with visual features via the latent semantic topics. Our proposed approach allows us to effectively annotate and refine tags based on a Monte Carlo Markov Chain algorithm for approximate inference. Moreover, we present a measuring scheme using the refined tags and extracted topics for ranking the images. The experimental results from two large benchmark datasets show that our approach provides promising accuracy.

Featured correspondence topic model for semantic search on social image collections

A new framework to retrieve semantically relevant images from the social database.Probabilistic topic model to predict the missing tags and remove the noisy ones.Two algorithms for the estimation of model parameters and tag correspondence.The scoring scheme relies on the fusion of visual and textual information.The outperformance of image annotation and retrieval to state-of-the-art methods. Nowadays, due to the rapid growth of digital technologies, huge volumes of image data are created and shared on social media sites. User-provided tags attached to each social image are widely recognized as a bridge to fill the semantic gap between low-level image features and high-level concepts. Hence, a combination of images along with their corresponding tags is useful for intelligent retrieval systems, those are designed to gain high-level understanding from images and facilitate semantic search. However, user-provided tags in practice are usually incomplete and noisy, which may degrade the retrieval performance. To tackle this problem, we present a novel retrieval framework that automatically associates the visual content with textual tags and enables effective image search. To this end, we first propose a probabilistic topic model learned on social images to discover latent topics from the co-occurrence of tags and image features. Moreover, our topic model is built by exploiting the expert knowledge about the correlation between tags with visual contents and the relationship among image features that is formulated in terms of spatial location and color distribution. The discovered topics then help to predict missing tags of an unseen image as well as the ones partially labeled in the database. These predicted tags can greatly facilitate the reliable measure of semantic similarity between the query and database images. Therefore, we further present a scoring scheme to estimate the similarity by fusing textual tags and visual representation. Extensive experiments conducted on three benchmark datasets show that our topic model provides the accurate annotation against the noise and incompleteness of tags. Using our generalized scoring scheme, which is particularly advantageous to many types of queries, the proposed approach also outperforms state-of-the-art approaches in terms of retrieval accuracy.

Efficient Content-based Image Retrieval for Multi-Image Queries

With explosive growth of digital images, content-based image retrieval has been emerged as an active research topic with many opportunities and challenges. Most of the visual search systems focus on the task of finding specific object, but a little attention has been paid for multiple object retrieval. In this paper, we present an efficient framework for processing multi-image queries, where the users can input multiple images with their objects of interest. We first propose a number of methods to encode image representation of queries. This encoded representation can be indexed further to retrieve an initial list of candidates. Moreover, we present the measuring scheme using Geometric Verification to estimate the similarity between query and candidate images during re-ranking stage. The experimental results performed on a benchmark dataset show that our approach provides promising performance.

Improvement of image representation for large-scale image retrieval

Nowadays, evolution of multimedia and networking technologies makes demand for searching information is increasing expressively. Many applications have been developed for recognition tasks. In this paper, we present the methods to compute discriminative and low-dimensional image representation which plays an important role to improve performance of large-scale retrieval systems. We first propose two effective approaches to aggregate local features of each image into a single vector, which can overcome limitation of existing aggregation methods and make image representation more compact and discriminative. We then employ Laplacian Eigenmaps which can explicitly exploit neighborhood structure of data and propose a robust variant to reduce dimension of the aggregated vector. The experiments show that our approach provides competitive performance with fewer memory storage for each image representation, which can be suitable for most of indexing schemes and work efficiently with large-scale database.