David Liu

Integrated feature selection and higher-order spatial feature extraction for object categorization

In computer vision, the bag-of-visual words image representation has been shown to yield good results. Recent work has shown that modeling the spatial relationship between visual words further improves performance. Previous work extracts higher-order spatial features exhaustively. However, these spatial features are expensive to compute. We propose a novel method that simultaneously performs feature selection and feature extraction. Higher-order spatial features are progressively extracted based on selected lower order ones, thereby avoiding exhaustive computation. The method can be based on any additive feature selection algorithm such as boosting. Experimental results show that the method is computationally much more efficient than previous approaches, without sacrificing accuracy.

Unsupervised Image Categorization and Object Localization using Topic Models and Correspondences between Images

Topic models from the text understanding literature have shown promising results in unsupervised image categorization and object localization. Categories are treated as topics, and words are formed by vector quantizing local descriptors of image patches. Limitations of topic models include their weakness in localizing objects, and the requirement of a fairly large proportion of words coming from the object. We present a new approach that employs correspondences between images to provide information about object configuration, which in turn enhances the reliability of object localization and categorization. This approach is efficient, as it requires only a small number of correspondences. We show improved categorization and localization performance on real and synthetic data. Moreover, we can push the limits of topic models when the proportion of words coming from the object is very low.

Semantic-Shift for Unsupervised Object Detection

The bag of visual words representation has attracted a lot of attention in the computer vision community. In particular, Probabilistic Latent Semantic Analysis (PLSA) has been applied to object recognition as an unsupervised technique built on top of the bag of visual words representation. PLSA, however, does not explicitly consider the spatial information of the visual words. In this paper, we propose an iterative technique, where a modified form of PLSA provides location and scale estimates of the foreground object through the estimated latent semantic. In return, the updated location and scale estimates will improve the estimate of the latent semantic. We call this iterative algorithm Semantic-Shift. We show results with significant improvements over PLSA.

A Topic-Motion Model for Unsupervised Video Object Discovery

The bag-of-words representation has attracted a lot of attention recently in the field of object recognition. Based on the bag-of-words representation, topic models such as probabilistic latent semantic analysis (PLSA) have been applied to unsupervised object discovery in still images. In this paper, we extend topic models from still images to motion videos with the integration of a temporal model. We propose a novel spatial-temporal framework that uses topic models for appearance modeling, and the probabilistic data association (PDA) filter for motion modeling. The spatial and temporal models are tightly integrated so that motion ambiguities can be resolved by appearance, and appearance ambiguities can be resolved by motion. We show promising results that cannot be achieved by appearance or motion modeling alone.

DISCOV: A Framework for Discovering Objects in Video

This paper presents a probabilistic framework for discovering objects in video. The video can switch between different shots, the unknown objects can leave or enter the scene at multiple times, and the background can be cluttered. The framework consists of an appearance model and a motion model. The appearance model exploits the consistency of object parts in appearance across frames. We use maximally stable extremal regions as observations in the model and hence provide robustness to object variations in scale, lighting and viewpoint. The appearance model provides location and scale estimates of the unknown objects through a compact probabilistic representation. The compact representation contains knowledge of the scene at the object level, thus allowing us to augment it with motion information using a motion model. This framework can be applied to a wide range of different videos and object types, and provides a basis for higher level video content analysis tasks. We present applications of video object discovery to video content analysis problems such as video segmentation and threading, and demonstrate superior performance to methods that exploit global image statistics and frequent itemset data mining techniques.

Video retrieval based on object discovery

State-of-the-art video retrieval methods use global image statistics to provide low level descriptors or use object recognizers to provide high level features. Using global image statistics can be hindered by lack of explicitly characterizing the object of interest hence prone to retrieving irrelevant results, while using object recognizers can suffer from having to train a large number of object recognizers for different types of objects. We present a novel framework for content-based video retrieval. We use an unsupervised learning method to automatically discover and locate the object of interest in a video clip. This unsupervised learning algorithm alleviates the need for training a large number of object recognizers. Regional image characteristics are extracted from the object of interest to form a set of descriptors for each video. A novel ensemble-based matching algorithm compares the similarity between two videos based on the set of descriptors each video contains. Videos containing large pose, size, and lighting variations are used to validate our approach.

Soft shape context for iterative closest point registration

This paper introduces a shape descriptor, the soft shape context, motivated by the shape context method. Unlike the original shape context method, where each image point was hard assigned into a single histogram bin, we instead allow each image point to contribute to multiple bins, hence more robust to distortions. The soft shape context can easily be integrated into the iterative closest point (ICP) method as an auxiliary feature vector, enriching the representation of an image point from spatial information only, to spatial and shape information. This yields a registration method more robust than the original ICP method. The method is general for 2D shapes. It does not calculate derivatives, hence being able to handle shapes with junctions and discontinuities. We present experimental results to demonstrate the robustness compared with the standard ICP method.

Content-Free Image Retrieval using Bayesian Product Rule

Content-free image retrieval uses accumulated user feedback records to retrieve images without analyzing image pixels. We present a Bayesian-based algorithm to analyze user feedback and show that it outperforms a recent maximum entropy content-free algorithm, according to extensive experiments on trademark logo and 3D model datasets. The proposed algorithm also has the advantage of being applicable to both content-free and traditional content-based image retrieval, thus providing a common framework for these two paradigms

Robust real-time 3D trajectory tracking algorithms for visual tracking using weak perspective projection

In this paper, motion estimation algorithms for the most general tracking situation are developed. The proposed motion estimation algorithms are used to predict the location of target and then to generate a feasible control input so as to keep the target stationary in the center of the image. The work differs from the previous algorithm of motion estimation in that it is capable of decoupling the estimation of motion from the estimation of structure. The weak perspective projection is used to solve this problem. The modified optical flow is first calculated and then fed to motion estimation algorithms so as to generate an appropriate camera motion that achieves tracking. The important contribution of this work is that simple, numerically stable, none computation intensive, and correspondence-free 3D motion estimation algorithms are derived. A visual tracking system can be easily implemented and run in real-time due to the simplicity of the proposed algorithms and thus increases their efficiency. The robustness and feasibility of the proposed algorithms has been validated by a number of experiments.

Visual Tracking in Cluttered Environments Using the Visual Probabilistic Data Association Filter

Visual tracking in cluttered environments is attractive and challenging. This paper establishes a probabilistic framework, called the visual probabilistic data-association filter (VPDAF), to deal with this problem. The algorithm is based on the probabilistic data-association method for estimating a true target from a cluster of measurements. There are two other key concepts which are involved in VPDAF. First, the sensor data are visual, similar to the target in the image space, which is a crucial property that should not be ignored in target estimation. Second, the traditional probabilistic data-association filter for the underlying application is vulnerable to stationary disturbances in image space, mainly due to some annoying background scenes which are rather similar to the target. Intuitively, such persistent noises should be separated out and cleared away from the continuous measurement data for seeking successful target detection. The proposed VPDAF framework, which incorporates template matching, can achieve the goal of reliable realtime visual tracking. To demonstrate the superiority of the system performance, extensive yet challenging experiments have been conducted