Jen-Hao Hsiao

Deep learning of binary hash codes for fast image retrieval

Approximate nearest neighbor search is an efficient strategy for large-scale image retrieval. Encouraged by the recent advances in convolutional neural networks (CNNs), we propose an effective deep learning framework to generate binary hash codes for fast image retrieval. Our idea is that when the data labels are available, binary codes can be learned by employing a hidden layer for representing the latent concepts that dominate the class labels. The utilization of the CNN also allows for learning image representations. Unlike other supervised methods that require pair-wised inputs for binary code learning, our method learns hash codes and image representations in a point-wised manner, making it suitable for large-scale datasets. Experimental results show that our method outperforms several state-of-the-art hashing algorithms on the CIFAR-10 and MNIST datasets. We further demonstrate its scalability and efficacy on a large-scale dataset of 1 million clothing images.

A New Approach to Image Copy Detection Based on Extended Feature Sets

Conventional image copy detection research concentrates on finding features that are robust enough to resist various kinds of image attacks. However, finding a globally effective feature is difficult and, in many cases, domain dependent. Instead of simply extracting features from copyrighted images directly, we propose a new framework called the extended feature set for detecting copies of images. In our approach, virtual prior attacks are applied to copyrighted images to generate novel features, which serve as training data. The copy-detection problem can be solved by learning classifiers from the training data, thus, generated. Our approach can be integrated into existing copy detectors to further improve their performance. Experiment results demonstrate that the proposed approach can substantially enhance the accuracy of copy detection.

Rapid Clothing Retrieval via Deep Learning of Binary Codes and Hierarchical Search

This paper deals with the problem of clothing retrieval in a recommendation system. We develop a hierarchical deep search framework to tackle this problem. We use a pre-trained network model that has learned rich mid-level visual representations in module 1. Then, in module 2, we add a latent layer to the network and have neurons in this layer to learn hashes-like representations while fine-tuning it on the clothing dataset. Finally, module 3 achieves fast clothing retrieval using the learned hash codes and representations via a coarse-to-fine strategy. We use a large clothing dataset where 161,234 clothes images are collected and labeled. Experiments demonstrate the potential of our proposed framework for clothing retrieval in a large corpus.

Image Copy Detection via Grouping in Feature Space Based on Virtual Prior Attacks

In the past, many researches on image copy detection focused on finding a feature that is robust enough for various kinds of image attacks. But it is difficult to find a globally effective feature that is appropriate for many situations. In this paper, we introduce a classification framework to this problem, instead of solving the feature-selection problem. In our approach, novel features are generated by applying virtual prior attacks to copyrighted images, and the copy-detection problem is converted to a classification one that is more robust to solve. Our approach can combine existing image copy detectors and further raise their performances.

Intention-focused active reranking for image object retrieval

We consider the problem of ranking refinement for image object retrieval, whose goal is to improve an existing ranking function by a small number of labeled instances. To retrieve the relevant image object, one state-of-the-art approach is to use the relevance feedback: it first ranks the images in database based on a given ranking function (i.e., base ranker), and then rerank the initial result by further introducing users feedback information. The key challenge of combining the information from the base ranker and users feedback comes from the fact that the base ranker tends to give an imperfect result and the information obtained from users feedback tends to be very noisy. This paper describes an Intention-Focused Active Reranking, an approach for automatically finding the right information to re-estimate the query model. Three novel strategies are proposed to boost the performance of the base ranker: (1) an active selection criterion, which obtains a small number of feedback images that are the most informative to the base ranker for user labeling; (2) the user intention verification, which captures the users intention in object level to alleviate the query drift problem; (3) a discriminative query model re-estimation, which augments the generative approach with a model of the discriminative information conveyed by positive and negative feedback information. Experiments on a real world data set demonstrate the effectiveness of the proposed approach and furthermore it significantly outperforms the baseline visual bag-of-words retrieval.

Constructing a wrapper-based DRM system for digital content protection in digital libraries

Conventional digital libraries utilize access control and digital watermarking techniques to protect their digital content. These methods, however, have drawbacks. First, after passing the identity authentication process, authorized users can easily redistribute the digital assets. Second, it is impractical to expect a digital watermarking scheme to prevent all kinds of attack. Thus, how to enforce property rights after digital content has been released to authorized users is a crucial and challenging issue. In this paper, we propose a wrapper-based approach to digital content protection that integrates digital watermarking, cryptography, information protection technology, and a rights model. In this rights enforcement environment, the behavior of all content players is monitored and digital content can only be accessed after certain usage rules have been satisfied. Furthermore, the proposed architecture can be easily integrated into any digital content player, or even existing DRM systems in digital libraries. With the protection of the proposed DRM system, the abuse of digital content can be drastically reduced.

Visual-word-based duplicate image search with pseudo-relevance feedback

We aim to improve the bag-of-visual-words (BOW) model for near-duplicate image retrieval, by introducing a more fine-grained pseudo-relevance feedback process. The BOW method is based on vector quantization of affine invariant descriptors of image patches. Despite its popularity and simplicity, the retrieval performance of BOW is often unsatisfactory due to the large and diverse variations of near-duplicate images. We thus propose an information-theoretic feedback framework that employs available cues in the search result to find more relevant duplicate images which are hard to retrieve by using conventional BOW approaches. Our algorithm is experimentally evaluated under a severely attacked image database, and shown to significantly improve the retrieval accuracy over a non-feedback baseline.

Protecting Digital Library Collections with Collaborative Web Image Copy Detection

Theres a pressing need for protecting digital library (DL) collections since digital objects can be easily copied, edited, and redistributed. Digital content protection is thus attracting attention. In this paper, we propose a peer-to-peer approach to collaborative Web image copy detection for protecting DL collections, which is helpful to public libraries with limited resources. We design a collaborative framework that incorporates multiple peers to share the loads of crawling and copy detection tasks. It facilitates better utilization of limited network and computing resources. Our experiment results demonstrate the efficacy and potential of the proposed approach. Further investigation is needed to verify its scalability.

Incremental Learning for Fine-Grained Image Recognition

This paper considers the problem of fine-grained image recognition with a growing vocabulary. Since in many real world applications we often have to add a new object category or visual concept with just a few images to learn from, it is crucial to develop a method that is able to generalize the recognition model from existing classes to new classes. Deep convolutional neural networks are capable of constructing powerful image representations; however, these networks usually rely on a logistic loss function that cannot handle the incremental learning problem. In this paper, we present a new method that can efficiently learn a new class given only a limited number of training examples, which we evaluate on the problems of food and clothing recognition. To illustrate the performance of our proposed method on the task of recognizing different kinds of food, when using only 1.3\% of training examples per category we achieved about 73\% of the performance (as measured by F1-score) compared to when using all available training data.

Effective content tracking for digital rights management in digital libraries

A usual way for content protection of digital libraries is to use digital watermarks and a DRM-based access-control environment. These methods, however, have limitations. Digital watermarks embedded in digital content could be removed by malicious users via post-processing, whereas DRM-based access-control solutions could be hacked. In this paper, we introduce a content tracking mechanism that we have built for multimedia-content near-replica detection as the second line of defense. The integrated framework aims to detect unlawful copyright infringements on the Internet, and combines the strengths of static rights enforcement and dynamic illegal content tracking. The issues of accuracy and huge computation cost in copy detection have been addressed by the introduced content-based techniques. Our experiments demonstrate the efficacy of proposed copy detector.