Weihua Xiong

RGBD salient object detection: A benchmark and algorithms

Although depth information plays an important role in the human vision system, it is not yet well-explored in existing visual saliency computational models. In this work, we first introduce a large scale RGBD image dataset to address the problem of data deficiency in current research of RGBD salient object detection. To make sure that most existing RGB saliency models can still be adequate in RGBD scenarios, we continue to provide a simple fusion framework that combines existing RGB-produced saliency with new depth-induced saliency, the former one is estimated from existing RGB models while the latter one is based on the proposed multi-contextual contrast model. Moreover, a specialized multi-stage RGBD model is also proposed which takes account of both depth and appearance cues derived from low-level feature contrast, mid-level region grouping and high-level priors enhancement. Extensive experiments show the effectiveness and superiority of our model which can accurately locate the salient objects from RGBD images, and also assign consistent saliency values for the target objects.

Salient Object Detection via Structured Matrix Decomposition

Low-rank recovery models have shown potential for salient object detection, where a matrix is decomposed into a low-rank matrix representing image background and a sparse matrix identifying salient objects. Two deficiencies, however, still exist. First, previous work typically assumes the elements in the sparse matrix are mutually independent, ignoring the spatial and pattern relations of image regions. Second, when the low-rank and sparse matrices are relatively coherent, e.g., when there are similarities between the salient objects and background or when the background is complicated, it is difficult for previous models to disentangle them. To address these problems, we propose a novel structured matrix decomposition model with two structural regularizations: (1) a tree-structured sparsity-inducing regularization that captures the image structure and enforces patches from the same object to have similar saliency values, and (2) a Laplacian regularization that enlarges the gaps between salient objects and the background in feature space. Furthermore, high-level priors are integrated to guide the matrix decomposition and boost the detection. We evaluate our model for salient object detection on five challenging datasets including single object, multiple objects and complex scene images, and show competitive results as compared with 24 state-of-the-art methods in terms of seven performance metrics.Low-rank recovery models have shown potential for salient object detection, where a matrix is decomposed into a low-rank matrix representing image background and a sparse matrix identifying salient objects. Two deficiencies, however, still exist. First, previous work typically assumes the elements in the sparse matrix are mutually independent, ignoring the spatial and pattern relations of image regions. Second, when the low-rank and sparse matrices are relatively coherent, e.g., when there are similarities between the salient objects and background or when the background is complicated, it is difficult for previous models to disentangle them. To address these problems, we propose a novel structured matrix decomposition model with two structural regularizations: (1) a tree-structured sparsity-inducing regularization that captures the image structure and enforces patches from the same object to have similar saliency values, and (2) a Laplacian regularization that enlarges the gaps between salient objects and the background in feature space. Furthermore, high-level priors are integrated to guide the matrix decomposition and boost the detection. We evaluate our model for salient object detection on five challenging datasets including single object, multiple objects and complex scene images, and show competitive results as compared with 24 state-of-the-art methods in terms of seven performance metrics.

Illumination estimation via thin-plate spline interpolation.

Thin-plate spline interpolation is used to interpolate the chromaticity of the color of the incident scene illumination across a training set of images. Given the image of a scene under unknown illumination, the chromaticity of the scene illumination can be found from the interpolated function. The resulting illumination-estimation method can be used to provide color constancy under changing illumination conditions and automatic white balancing for digital cameras. A thin-plate spline interpolates over a nonuniformly sampled input space, which in this case is a training set of image thumbnails and associated illumination chromaticities. To reduce the size of the training set, incremental k medians are applied. Tests on real images demonstrate that the thin-plate spline method can estimate the color of the incident illumination quite accurately, and the proposed training set pruning significantly decreases the computation.

Evaluating Combinational Illumination Estimation Methods on Real-World Images

Illumination estimation is an important component of color constancy and automatic white balancing. A number of methods of combining illumination estimates obtained from multiple subordinate illumination estimation methods now appear in the literature. These combinational methods aim to provide better illumination estimates by fusing the information embedded in the subordinate solutions. The existing combinational methods are surveyed and analyzed here with the goals of determining: 1) the effectiveness of fusing illumination estimates from multiple subordinate methods; 2) the best method of combination; 3) the underlying factors that affect the performance of a combinational method; and 4) the effectiveness of combination for illumination estimation in multiple-illuminant scenes. The various combinational methods are categorized in terms of whether or not they require supervised training and whether or not they rely on high-level scene content cues (e.g., indoor versus outdoor). Extensive tests and enhanced analyzes using three data sets of real-world images are conducted. For consistency in testing, the images were labeled according to their high-level features (3D stages, indoor/outdoor) and this label data is made available on-line. The tests reveal that the trained combinational methods (direct combination by support vector regression in particular) clearly outperform both the non-combinational methods and those combinational methods based on scene content cues.

A supervised combination strategy for illumination chromaticity estimation

Color constancy is an important perceptual ability of humans to recover the color of objects invariant of light information. It is also necessary for a robust machine vision system. Until now, a number of color constancy algorithms have been proposed in the literature. In particular, the edge-based color constancy uses the edge of an image to estimate light color. It is shown to be a rich framework that can represent many existing illumination estimation solutions with various parameter settings. However, color constancy is an ill-posed problem; every algorithm is always given out under some assumptions and can only produce the best performance when these assumptions are satisfied. In this article, we have investigated a combination strategy relying on the Extreme Learning Machine (ELM) technique that integrates the output of edge-based color constancy with multiple parameters. Experiments on real image data sets show that the proposed method works better than most single-color constancy methods and even some current state-of-the-art color constancy combination strategies.

Hierarchical sparse representation based Multi-Instance Semi-Supervised Learning with application to image categorization

Recent studies have shown that sparse representation (SR) can deal well with many computer vision problems. In this paper, we extend a hierarchical sparse representation algorithm into Multi-Instance Semi-Supervised Learning (MISSL) problem. Specifically, at the instance level, after investigating the properties of true positive instances in depth, we propose a novel instance disambiguation strategy based on sparse representation that can identify the instance confidence value in both positive and unlabeled bags more effectively. At the bag level, in contrast to the traditional k-NN or @e-graph construction methods used in the graph-based semi-supervised learning settings, we propose a weighted multi-instance kernel and a corresponding kernel sparse representation method for robust @?^1-graph construction. The improved @?^1-graph that encodes the multi-instance properties can be utilized in the manifold regularization framework for the label propagation. Experimental results on different image data sets have demonstrated that the proposed algorithm outperforms existing multi-instance learning (MIL) algorithms, as well as the MISSL algorithms with the application to image categorization task.

Multi-Cue Illumination Estimation via a Tree-Structured Group Joint Sparse Representation

A multi-cue illumination estimation method based on tree-structured group joint sparse representation is proposed. Tests show that the proposed method works better than existing methods, most of which are based on using only a single cue type, for example, a binarized color histogram or simple image statistic such as the mean RGB. Most existing illumination estimation methods make their estimates using only one of three kinds of cues. They differ in which cue type they use, but the chosen cue is either based on (1) properties of the low-level RGB color distribution, (2) mid-level initial illuminant estimates provided by subordinate methods, or (3) high-level knowledge of scene content (e.g., indoor versus outdoor scene). The proposed multi-cue method combines the information provided by cues of all three of these types within the framework of a tree-structured group joint sparse representation (TGJSR). In TGJSR, the training data is grouped into a tree of subgroups. A test image under an unknown illuminant has its features reconstructed in terms of a joint sparse representation model derived from the grouped training data. The test image’s illumination is then estimated based on the weights involved in the joint sparse representation model. As a general framework, the proposed TGJSR framework can also easily be extended to incorporate any new features or cues that might be discovered in the future for illumination estimation.

Evaluating combinational color constancy methods on real-world images

Light color estimation is crucial to the color constancy problem. Past decades have witnessed great progress in solving this problem. Contrary to traditional methods, many researchers propose a variety of combinational color constancy methods through applying different color constancy mathematical models on an image simultaneously and then give out a final estimation in diverse ways. Although many comprehensive evaluations or reviews about color constancy methods are available, few focus on combinational strategies. In this paper, we survey some prevailing combinational strategies systematically; divide them into three categories and compare them qualitatively on three real-world image data sets in terms of the angular error and the perceptual Euclidean distance. The experimental results show that combinational strategies with training procedure always produces better performance.

Horror video scene recognition based on multi-view multi-instance learning

Comparing with the research of pornographic content filtering on Web, Web horror content filtering, especially horror video scene recognition is still on the stage of exploration. Most existing methods identify horror scene only from independent frames, ignoring the context cues among frames in a video scene. In this paper, we propose a Multi-view Multi-Instance Leaning (M2IL) model based on joint sparse coding technique that takes the bag of instances from independent view and contextual view into account simultaneously and apply it on horror scene recognition. Experiments on a horror video dataset collected from internet demonstrate that our methods performance is superior to the other existing algorithms.

Multi-Instance Multi-Label Learning Combining Hierarchical Context and its Application to Image Annotation

In image annotation, one image is often modeled as a bag of regions (“instances”) associated with multiple labels, which is a typical application of multi-instance multi-label learning (MIML). Although lots of research has shown that the interplay embedded among instances and labels can largely boost the image annotation accuracy, most existing MIML methods consider none or partial context cues. In this paper, we propose a novel context-aware MIML model to integrate the instance context and label context into a general framework. Specially, the instance context is constructed with multiple graphs, while the label context is built up through a linear combination of several common latent conceptions that link low level features and high level semantic labels. Comparison with other leading methods on several benchmark datasets in terms of image annotation shows that our proposed method can get better performance than the state-of-the-art approaches.