Guihua Er

Similarity-based online feature selection in content-based image retrieval

Content-based image retrieval (CBIR) has been more and more important in the last decade, and the gap between high-level semantic concepts and low-level visual features hinders further performance improvement. The problem of online feature selection is critical to really bridge this gap. In this paper, we investigate online feature selection in the relevance feedback learning process to improve the retrieval performance of the region-based image retrieval system. Our contributions are mainly in three areas. 1) A novel feature selection criterion is proposed, which is based on the psychological similarity between the positive and negative training sets. 2) An effective online feature selection algorithm is implemented in a boosting manner to select the most representative features for the current query concept and combine classifiers constructed over the selected features to retrieve images. 3) To apply the proposed feature selection method in region-based image retrieval systems, we propose a novel region-based representation to describe images in a uniform feature space with real-valued fuzzy features. Our system is suitable for online relevance feedback learning in CBIR by meeting the three requirements: learning with small size training set, the intrinsic asymmetry property of training samples, and the fast response requirement. Extensive experiments, including comparisons with many state-of-the-arts, show the effectiveness of our algorithm in improving the retrieval performance and saving the processing time.

A Novel Method for Semi-automatic 2D to 3D Video Conversion

In this paper, we present a novel semi-automatic method for converting monoscopic video to stereoscopic video. An efficient interactive image cutout tool is first used to segment the object-of-interest in the key frames. Then, we assign the initial depth information to the segmented objects. These objects are tracked in the whole video sequence through a bi-directional KIT (Kanade-Iucas- Tomashi) algorithm. In addition, depth interpolation is employed to produce the depth information in the non-key frames. Finally, stereoscopic video is synthesized in consideration of different 3D display types. The experimental results show that our method can fulfill 2D to 3D video conversion both reliably and efficiently.

Weighted Subspace Distance and Its Applications to Object Recognition and Retrieval With Image Sets

We address the problem of measuring the distance between two subspaces, each of which is spanned by an image set. In the existing methods, only the orthonormal basis is used to represent the subspace. However, the images are usually distributed in a limited area, rather than the whole subspace. Therefore, the characteristics of the distribution should also be considered. In this letter, a weighted subspace distance (WSD) is proposed, in which the principal component values of the data set are adopted to calculate the weights. Experimental results on object recognition and retrieval with image sets demonstrate the effectiveness of our proposal.

Multilabel Neighborhood Propagation for Region-Based Image Retrieval

Content-based image retrieval (CBIR) has been an active research topic in the last decade. As one of the promising approaches, graph-based semi-supervised learning has attracted many researchers. However, while the related work mainly focused on global visual features, little attention has been paid to region-based image retrieval (RBIR). In this paper, a framework based on multilabel neighborhood propagation is proposed for RBIR, which can be characterized by three key properties: (1) For graph construction, in order to determine the edge weights robustly and automatically, mixture distribution is introduced into the Earth movers distance (EMD) and a linear programming framework is involved. (2) Multiple low-level labels for each image can be obtained based on a generative model, and the correlations among different labels are explored when the labels are propagated simultaneously on the weighted graph. (3) By introducing multilayer semantic representation (MSR) and support vector machine (SVM) into the long-term learning, more exact weighted graph for label propagation and more meaningful high-level labels to describe the images can be calculated. Experimental results, including comparisons with the state-of-the-art retrieval systems, demonstrate the effectiveness of our proposal.

Quality assessment of 3D asymmetric view coding using spatial frequency dominance model

To save bit-rate in stereo video application, asymmetric view coding is introduced, which encodes the stereo views with different qualities. However, quality assessment on asymmetric view coding is difficult, because the impact of the degraded view upon the 3D percept depends on Human Visual System (HVS) and cannot be indicated by conventional metrics. This paper introduces a quality assessment model based on the observed phenomenon that spatial frequency determines view domination under the action of HVS. A metric is proposed based on this model for assessing the quality of asymmetric view coding. Experimental results are presented to show that the proposed metric provides accordant assessment with the subjective evaluation.

Depth map generation for 2D-to-3D conversion by limited user inputs and depth propagation

The quality of depth maps is crucial for 2D to 3D conversion, but high quality depth map generation methods are usually very time consuming. We propose an efficient semi-automatic depth map generation scheme based on limited user inputs and depth propagation. First, the original image is over-segmented. Then, the depth values of selected pixels and the approximate locations of T-junctions are specified by user inputs. The final depth map is obtained by depth propagation combining user inputs, color and edge information. The experimental results demonstrate that our scheme is satisfactory in terms of both accuracy and efficiency, and thus can be applied for high quality 2D to 3D video conversion.

2D-to-3D Conversion Based on Motion and Color Mergence

In this paper, we present an efficient scheme to synthesize stereoscopic video from monoscopic video. We use the improved optical flow method to extract pixel-level motion for each frame. By considering the intensity of the estimated motion, we can classify the moving objects. Then, to achieve more accurate classification, we combine color information in the frame using the method derives from the minimum discrimination information (MDI) principle. Finally, constraints-involved flood-fill method is developed to segment the frame and assign depth values for different segmented regions. The experimental results show that our scheme achieves good performances on both segmentation and depth determination.

Hidden annotation for image retrieval with long-term relevance feedback learning

Hidden annotation (HA) is an important research issue in content-based image retrieval (CBIR). We propose to incorporate long-term relevance feedback (LRF) with HA to increase both efficiency and retrieval accuracy of CBIR systems. The work contains two parts. (1) Through LRF, a multi-layer semantic representation is built to automatically extract hidden semantic concepts underlying images. HA with these concepts alleviates the burden of manual annotation and avoids the ambiguity problem of keyword-based annotation. (2) For each learned concept, semi-supervised learning is incorporated to automatically select a small number of candidate images for annotators to annotate, which improves efficiency of HA.

Statistical modeling and many-to-many matching for view-based 3D object retrieval

We address the task of view-based 3D object retrieval, in which each object is represented by a set of views taken from different positions, rather than a geometrical model based on polygonal meshes. As the number of views and the view point setting cannot always be the same for different objects, the retrieval task is more challenging and the existing methods for 3D model retrieval are infeasible. In this paper, the information in the sets of views is exploited from two aspects. On the one hand, the form of histogram is converted from vector to state sequence, and Markov chain (MC) is utilized for modeling the statistical characteristics of all the views representing the same object. On the other hand, the earth movers distance (EMD) is involved to achieve many-to-many matching between two sets of views. For 3D object retrieval, by combining the above two aspects together, a new distance measure is defined, and a novel approach to automatically determine the edge weights in graph-based semi-supervised learning is proposed. Experimental results on different databases demonstrate the effectiveness of our proposal.

Histogram mining based on Markov chain and its application to image categorization

Histogram is a useful feature for image content analysis and has been widely used in many methods for image categorization. Most of the existing classifiers usually cannot distinguish the effects of different bins in histogram, except for setting different weights. However, these weights are often difficult to be exactly determined in advance. To further mine the information in histogram, in this paper, we propose a method to represent the histogram in another form called quasi-histogram, which can be thought as the state sequence of a Markov chain (MC). By modeling the quasi-histogram of each image as having been stochastically generated by an MC corresponding to its category, we can take the characteristic of each bin into account. Improved image categorization performance can be obtained through combining the results of the traditional classifier with those of MC. Experimental results show the effectiveness of our proposal.